Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2200,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2200,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.741315
Epoch 1
Validation auc = 0.779812
Epoch 2
Validation auc = 0.757746
Epoch 3
Validation auc = 0.716901
Epoch 4
Loss = 5.6574e-01, PNorm = 86.2021, GNorm = 9.7580, lr_0 = 6.5000e-04
Validation auc = 0.693427
Epoch 5
Validation auc = 0.692019
Epoch 6
Validation auc = 0.684977
Epoch 7
Validation auc = 0.651643
Epoch 8
Validation auc = 0.678873
Epoch 9
Loss = 2.0507e-01, PNorm = 86.4871, GNorm = 3.5668, lr_0 = 9.9743e-04
Validation auc = 0.704695
Epoch 10
Validation auc = 0.729577
Epoch 11
Validation auc = 0.748826
Epoch 12
Validation auc = 0.752113
Epoch 13
Validation auc = 0.686385
Epoch 14
Loss = 3.4532e-01, PNorm = 86.7652, GNorm = 3.7138, lr_0 = 9.8890e-04
Validation auc = 0.666667
Epoch 15
Validation auc = 0.694366
Epoch 16
Validation auc = 0.721127
Epoch 17
Validation auc = 0.722066
Epoch 18
Validation auc = 0.729577
Epoch 19
Loss = 3.3321e-01, PNorm = 86.9568, GNorm = 3.2045, lr_0 = 9.8045e-04
Validation auc = 0.735681
Epoch 20
Validation auc = 0.724413
Epoch 21
Validation auc = 0.689671
Epoch 22
Validation auc = 0.670423
Epoch 23
Validation auc = 0.667606
Epoch 24
Loss = 1.3997e-01, PNorm = 87.1173, GNorm = 3.0361, lr_0 = 9.7207e-04
Validation auc = 0.670892
Epoch 25
Validation auc = 0.674178
Epoch 26
Validation auc = 0.677465
Epoch 27
Validation auc = 0.707042
Epoch 28
Validation auc = 0.706103
Epoch 29
Loss = 1.5928e-01, PNorm = 87.2666, GNorm = 4.4207, lr_0 = 9.6376e-04
Validation auc = 0.732394
Epoch 30
Validation auc = 0.709859
Epoch 31
Validation auc = 0.704695
Epoch 32
Validation auc = 0.684977
Epoch 33
Validation auc = 0.704225
Epoch 34
Loss = 1.3662e-01, PNorm = 87.4314, GNorm = 3.7869, lr_0 = 9.5552e-04
Validation auc = 0.707981
Epoch 35
Validation auc = 0.701878
Epoch 36
Validation auc = 0.691549
Epoch 37
Validation auc = 0.669953
Epoch 38
Validation auc = 0.651174
Epoch 39
Loss = 1.3532e-01, PNorm = 87.5658, GNorm = 5.8721, lr_0 = 9.4735e-04
Validation auc = 0.665258
Epoch 40
Validation auc = 0.679812
Epoch 41
Validation auc = 0.687324
Epoch 42
Validation auc = 0.679812
Epoch 43
Validation auc = 0.704225
Epoch 44
Loss = 5.0124e-02, PNorm = 87.6745, GNorm = 1.6685, lr_0 = 9.3925e-04
Validation auc = 0.715023
Epoch 45
Validation auc = 0.675587
Epoch 46
Validation auc = 0.684038
Epoch 47
Validation auc = 0.688263
Epoch 48
Validation auc = 0.693897
Epoch 49
Loss = 6.4708e-02, PNorm = 87.7610, GNorm = 1.3387, lr_0 = 9.3122e-04
Validation auc = 0.696244
Epoch 50
Validation auc = 0.681221
Epoch 51
Validation auc = 0.682629
Epoch 52
Validation auc = 0.668075
Epoch 53
Validation auc = 0.670423
Epoch 54
Loss = 1.2578e-01, PNorm = 87.8792, GNorm = 3.2288, lr_0 = 9.2326e-04
Validation auc = 0.673709
Epoch 55
Validation auc = 0.674648
Epoch 56
Validation auc = 0.685915
Epoch 57
Validation auc = 0.692958
Epoch 58
Validation auc = 0.705164
Epoch 59
Loss = 6.2513e-02, PNorm = 87.9603, GNorm = 2.4025, lr_0 = 9.1537e-04
Validation auc = 0.713615
Epoch 60
Validation auc = 0.721596
Epoch 61
Validation auc = 0.716432
Epoch 62
Validation auc = 0.695775
Epoch 63
Validation auc = 0.695775
Epoch 64
Loss = 2.2236e-01, PNorm = 88.0280, GNorm = 10.7722, lr_0 = 9.0754e-04
Validation auc = 0.692488
Epoch 65
Validation auc = 0.705164
Epoch 66
Validation auc = 0.727700
Epoch 67
Validation auc = 0.737559
Epoch 68
Validation auc = 0.758685
Epoch 69
Loss = 4.3561e-02, PNorm = 88.1367, GNorm = 1.5620, lr_0 = 8.9978e-04
Validation auc = 0.762911
Epoch 70
Validation auc = 0.730986
Epoch 71
Validation auc = 0.693427
Epoch 72
Validation auc = 0.682160
Epoch 73
Validation auc = 0.690141
Epoch 74
Loss = 4.5232e-02, PNorm = 88.3231, GNorm = 0.7980, lr_0 = 8.9209e-04
Validation auc = 0.699061
Epoch 75
Validation auc = 0.696244
Epoch 76
Validation auc = 0.678873
Epoch 77
Validation auc = 0.663380
Epoch 78
Validation auc = 0.656338
Epoch 79
Loss = 7.4037e-02, PNorm = 88.4499, GNorm = 1.4818, lr_0 = 8.8447e-04
Validation auc = 0.650704
Epoch 80
Validation auc = 0.660094
Epoch 81
Validation auc = 0.672300
Epoch 82
Validation auc = 0.680751
Epoch 83
Validation auc = 0.692488
Epoch 84
Loss = 6.8529e-02, PNorm = 88.5701, GNorm = 1.2309, lr_0 = 8.7691e-04
Validation auc = 0.697653
Epoch 85
Validation auc = 0.702347
Epoch 86
Validation auc = 0.698592
Epoch 87
Validation auc = 0.704695
Epoch 88
Validation auc = 0.708920
Epoch 89
Loss = 3.0375e-02, PNorm = 88.6698, GNorm = 1.0215, lr_0 = 8.6941e-04
Validation auc = 0.702347
Epoch 90
Validation auc = 0.694836
Epoch 91
Validation auc = 0.684977
Epoch 92
Validation auc = 0.685915
Epoch 93
Validation auc = 0.700000
Epoch 94
Loss = 4.0501e-02, PNorm = 88.7389, GNorm = 0.3290, lr_0 = 8.6198e-04
Validation auc = 0.708451
Epoch 95
Validation auc = 0.708451
Epoch 96
Validation auc = 0.700939
Epoch 97
Validation auc = 0.695305
Epoch 98
Validation auc = 0.694366
Epoch 99
Loss = 7.7920e-02, PNorm = 88.7876, GNorm = 0.1433, lr_0 = 8.5461e-04
Validation auc = 0.697653
Epoch 100
Validation auc = 0.695775
Epoch 101
Validation auc = 0.689671
Epoch 102
Validation auc = 0.678404
Epoch 103
Validation auc = 0.668545
Epoch 104
Loss = 7.0569e-02, PNorm = 88.8345, GNorm = 3.6565, lr_0 = 8.4730e-04
Validation auc = 0.667136
Epoch 105
Validation auc = 0.686385
Epoch 106
Validation auc = 0.707042
Epoch 107
Validation auc = 0.723944
Epoch 108
Validation auc = 0.726761
Epoch 109
Loss = 5.4971e-02, PNorm = 88.8804, GNorm = 0.6881, lr_0 = 8.4006e-04
Validation auc = 0.717371
Epoch 110
Validation auc = 0.708920
Epoch 111
Validation auc = 0.707981
Epoch 112
Validation auc = 0.710798
Epoch 113
Validation auc = 0.700000
Epoch 114
Loss = 9.2417e-02, PNorm = 88.9304, GNorm = 0.8686, lr_0 = 8.3288e-04
Validation auc = 0.686854
Epoch 115
Validation auc = 0.678873
Epoch 116
Validation auc = 0.679812
Epoch 117
Validation auc = 0.683568
Epoch 118
Validation auc = 0.688732
Epoch 119
Loss = 1.5662e-02, PNorm = 88.9695, GNorm = 0.8041, lr_0 = 8.2576e-04
Validation auc = 0.695775
Epoch 120
Validation auc = 0.698592
Epoch 121
Validation auc = 0.700469
Epoch 122
Validation auc = 0.698122
Epoch 123
Validation auc = 0.693897
Epoch 124
Loss = 2.2249e-02, PNorm = 89.0016, GNorm = 0.2853, lr_0 = 8.1870e-04
Validation auc = 0.692488
Epoch 125
Validation auc = 0.688263
Epoch 126
Validation auc = 0.687793
Epoch 127
Validation auc = 0.697653
Epoch 128
Validation auc = 0.715493
Epoch 129
Loss = 8.0359e-02, PNorm = 89.0311, GNorm = 4.5348, lr_0 = 8.1170e-04
Validation auc = 0.721127
Epoch 130
Validation auc = 0.722535
Epoch 131
Validation auc = 0.721127
Epoch 132
Validation auc = 0.724883
Epoch 133
Validation auc = 0.725352
Epoch 134
Loss = 6.3750e-03, PNorm = 89.0617, GNorm = 0.5203, lr_0 = 8.0476e-04
Validation auc = 0.717840
Epoch 135
Validation auc = 0.718310
Epoch 136
Validation auc = 0.709859
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.706573
Epoch 139
Loss = 6.0801e-02, PNorm = 89.0930, GNorm = 4.8772, lr_0 = 7.9788e-04
Validation auc = 0.707512
Epoch 140
Validation auc = 0.697183
Epoch 141
Validation auc = 0.692958
Epoch 142
Validation auc = 0.686385
Epoch 143
Validation auc = 0.682160
Epoch 144
Loss = 2.8911e-02, PNorm = 89.1273, GNorm = 0.6845, lr_0 = 7.9106e-04
Validation auc = 0.678873
Epoch 145
Validation auc = 0.683099
Epoch 146
Validation auc = 0.688263
Epoch 147
Validation auc = 0.695775
Epoch 148
Validation auc = 0.698122
Epoch 149
Loss = 2.5875e-02, PNorm = 89.1649, GNorm = 1.7666, lr_0 = 7.8430e-04
Validation auc = 0.703286
Epoch 150
Validation auc = 0.702817
Epoch 151
Validation auc = 0.699061
Epoch 152
Validation auc = 0.702347
Epoch 153
Validation auc = 0.703286
Epoch 154
Loss = 2.7706e-02, PNorm = 89.1941, GNorm = 0.1381, lr_0 = 7.7759e-04
Validation auc = 0.694836
Epoch 155
Validation auc = 0.687793
Epoch 156
Validation auc = 0.682160
Epoch 157
Validation auc = 0.680751
Epoch 158
Validation auc = 0.676995
Epoch 159
Loss = 1.0441e-02, PNorm = 89.2161, GNorm = 0.1323, lr_0 = 7.7095e-04
Validation auc = 0.679343
Epoch 160
Validation auc = 0.681221
Epoch 161
Validation auc = 0.676995
Epoch 162
Validation auc = 0.676995
Epoch 163
Validation auc = 0.673709
Epoch 164
Loss = 3.3995e-02, PNorm = 89.2401, GNorm = 1.6481, lr_0 = 7.6436e-04
Validation auc = 0.677934
Epoch 165
Validation auc = 0.677934
Epoch 166
Validation auc = 0.678404
Epoch 167
Validation auc = 0.683568
Epoch 168
Validation auc = 0.692488
Epoch 169
Loss = 6.3709e-03, PNorm = 89.2701, GNorm = 0.0917, lr_0 = 7.5782e-04
Validation auc = 0.698122
Epoch 170
Validation auc = 0.702347
Epoch 171
Validation auc = 0.700939
Epoch 172
Validation auc = 0.691080
Epoch 173
Validation auc = 0.691080
Epoch 174
Loss = 6.5805e-02, PNorm = 89.3089, GNorm = 2.7112, lr_0 = 7.5134e-04
Validation auc = 0.684507
Epoch 175
Validation auc = 0.681221
Epoch 176
Validation auc = 0.682629
Epoch 177
Validation auc = 0.684507
Epoch 178
Validation auc = 0.688732
Epoch 179
Loss = 8.1824e-02, PNorm = 89.3504, GNorm = 3.3366, lr_0 = 7.4492e-04
Validation auc = 0.689202
Epoch 180
Validation auc = 0.692019
Epoch 181
Validation auc = 0.693427
Epoch 182
Validation auc = 0.694366
Epoch 183
Validation auc = 0.693897
Epoch 184
Loss = 3.8281e-03, PNorm = 89.3840, GNorm = 0.1400, lr_0 = 7.3855e-04
Validation auc = 0.696714
Epoch 185
Validation auc = 0.697653
Epoch 186
Validation auc = 0.692958
Epoch 187
Validation auc = 0.691080
Epoch 188
Validation auc = 0.687793
Epoch 189
Loss = 2.5494e-02, PNorm = 89.4203, GNorm = 3.0646, lr_0 = 7.3224e-04
Validation auc = 0.684507
Epoch 190
Validation auc = 0.685915
Epoch 191
Validation auc = 0.684038
Epoch 192
Validation auc = 0.683568
Epoch 193
Validation auc = 0.682160
Epoch 194
Loss = 1.1476e-02, PNorm = 89.4619, GNorm = 1.0258, lr_0 = 7.2598e-04
Validation auc = 0.684038
Epoch 195
Validation auc = 0.686854
Epoch 196
Validation auc = 0.690141
Epoch 197
Validation auc = 0.691549
Epoch 198
Validation auc = 0.694366
Epoch 199
Loss = 1.5625e-03, PNorm = 89.4907, GNorm = 0.0618, lr_0 = 7.1977e-04
Validation auc = 0.695775
Epoch 200
Validation auc = 0.696714
Epoch 201
Validation auc = 0.696244
Epoch 202
Validation auc = 0.696714
Epoch 203
Validation auc = 0.696244
Epoch 204
Loss = 2.5056e-02, PNorm = 89.5185, GNorm = 0.8361, lr_0 = 7.1362e-04
Validation auc = 0.693897
Epoch 205
Validation auc = 0.690610
Epoch 206
Validation auc = 0.691549
Epoch 207
Validation auc = 0.691549
Epoch 208
Validation auc = 0.695305
Epoch 209
Loss = 7.0650e-03, PNorm = 89.5553, GNorm = 0.5865, lr_0 = 7.0752e-04
Validation auc = 0.695775
Epoch 210
Validation auc = 0.697653
Epoch 211
Validation auc = 0.692019
Epoch 212
Validation auc = 0.694366
Epoch 213
Validation auc = 0.694366
Epoch 214
Loss = 3.9282e-03, PNorm = 89.6034, GNorm = 0.2010, lr_0 = 7.0147e-04
Validation auc = 0.692019
Epoch 215
Validation auc = 0.691080
Epoch 216
Validation auc = 0.695305
Epoch 217
Validation auc = 0.692019
Epoch 218
Validation auc = 0.689671
Epoch 219
Loss = 5.6268e-03, PNorm = 89.6431, GNorm = 0.1171, lr_0 = 6.9548e-04
Validation auc = 0.682629
Epoch 220
Validation auc = 0.681221
Epoch 221
Validation auc = 0.675587
Epoch 222
Validation auc = 0.676056
Epoch 223
Validation auc = 0.673709
Epoch 224
Loss = 8.2652e-03, PNorm = 89.6843, GNorm = 0.4233, lr_0 = 6.8953e-04
Validation auc = 0.684038
Epoch 225
Validation auc = 0.681690
Epoch 226
Validation auc = 0.689671
Epoch 227
Validation auc = 0.690141
Epoch 228
Validation auc = 0.696244
Epoch 229
Loss = 9.3678e-03, PNorm = 89.7731, GNorm = 1.2891, lr_0 = 6.8364e-04
Validation auc = 0.702347
Epoch 230
Validation auc = 0.707981
Epoch 231
Validation auc = 0.716432
Epoch 232
Validation auc = 0.711268
Epoch 233
Validation auc = 0.700000
Epoch 234
Loss = 4.3294e-03, PNorm = 89.8285, GNorm = 0.8849, lr_0 = 6.7779e-04
Validation auc = 0.693427
Epoch 235
Validation auc = 0.689671
Epoch 236
Validation auc = 0.686385
Epoch 237
Validation auc = 0.686385
Epoch 238
Validation auc = 0.685446
Epoch 239
Loss = 4.3998e-03, PNorm = 89.8684, GNorm = 0.8139, lr_0 = 6.7200e-04
Validation auc = 0.687324
Epoch 240
Validation auc = 0.687793
Epoch 241
Validation auc = 0.685446
Epoch 242
Validation auc = 0.684507
Epoch 243
Validation auc = 0.681690
Epoch 244
Loss = 1.8175e-03, PNorm = 89.9031, GNorm = 0.0969, lr_0 = 6.6625e-04
Validation auc = 0.680751
Epoch 245
Validation auc = 0.679812
Epoch 246
Validation auc = 0.692019
Epoch 247
Validation auc = 0.703756
Epoch 248
Validation auc = 0.718779
Epoch 249
Loss = 2.3015e-03, PNorm = 89.9542, GNorm = 0.0786, lr_0 = 6.6056e-04
Validation auc = 0.722066
Epoch 250
Validation auc = 0.727700
Epoch 251
Validation auc = 0.737559
Epoch 252
Validation auc = 0.742254
Epoch 253
Validation auc = 0.745070
Epoch 254
Loss = 2.8978e-03, PNorm = 89.9705, GNorm = 0.1823, lr_0 = 6.5491e-04
Validation auc = 0.744601
Epoch 255
Validation auc = 0.729577
Epoch 256
Validation auc = 0.714554
Epoch 257
Validation auc = 0.701878
Epoch 258
Validation auc = 0.700939
Epoch 259
Loss = 1.2150e-02, PNorm = 90.0113, GNorm = 1.0516, lr_0 = 6.4931e-04
Validation auc = 0.706573
Epoch 260
Validation auc = 0.707981
Epoch 261
Validation auc = 0.708451
Epoch 262
Validation auc = 0.708451
Epoch 263
Validation auc = 0.706103
Epoch 264
Loss = 1.0245e-02, PNorm = 90.0643, GNorm = 0.9286, lr_0 = 6.4376e-04
Validation auc = 0.705634
Epoch 265
Validation auc = 0.705164
Epoch 266
Validation auc = 0.703756
Epoch 267
Validation auc = 0.704695
Epoch 268
Validation auc = 0.701878
Epoch 269
Loss = 1.0262e-02, PNorm = 90.1096, GNorm = 1.0955, lr_0 = 6.3826e-04
Validation auc = 0.704695
Epoch 270
Validation auc = 0.706573
Epoch 271
Validation auc = 0.712207
Epoch 272
Validation auc = 0.713615
Epoch 273
Validation auc = 0.713146
Epoch 274
Loss = 7.4146e-03, PNorm = 90.1349, GNorm = 0.8612, lr_0 = 6.3280e-04
Validation auc = 0.715493
Epoch 275
Validation auc = 0.718310
Epoch 276
Validation auc = 0.717840
Epoch 277
Validation auc = 0.715962
Epoch 278
Validation auc = 0.714085
Epoch 279
Loss = 7.5157e-04, PNorm = 90.1631, GNorm = 0.0273, lr_0 = 6.2739e-04
Validation auc = 0.709859
Epoch 280
Validation auc = 0.708920
Epoch 281
Validation auc = 0.704695
Epoch 282
Validation auc = 0.705164
Epoch 283
Validation auc = 0.704695
Epoch 284
Loss = 4.2799e-03, PNorm = 90.1912, GNorm = 0.2059, lr_0 = 6.2203e-04
Validation auc = 0.704695
Epoch 285
Validation auc = 0.702817
Epoch 286
Validation auc = 0.702347
Epoch 287
Validation auc = 0.701408
Epoch 288
Validation auc = 0.706103
Epoch 289
Loss = 2.6593e-03, PNorm = 90.2168, GNorm = 0.1072, lr_0 = 6.1671e-04
Validation auc = 0.707042
Epoch 290
Validation auc = 0.710329
Epoch 291
Validation auc = 0.713615
Epoch 292
Validation auc = 0.712676
Epoch 293
Validation auc = 0.713615
Epoch 294
Loss = 2.0402e-03, PNorm = 90.2455, GNorm = 0.0331, lr_0 = 6.1144e-04
Validation auc = 0.713146
Epoch 295
Validation auc = 0.709390
Epoch 296
Validation auc = 0.707042
Epoch 297
Validation auc = 0.707512
Epoch 298
Validation auc = 0.709859
Epoch 299
Loss = 2.5014e-03, PNorm = 90.2823, GNorm = 0.0246, lr_0 = 6.0621e-04
Validation auc = 0.709859
Model 0 best validation auc = 0.779812 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.881999
Ensemble test auc = 0.881999
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.724883
Epoch 1
Validation auc = 0.777465
Epoch 2
Validation auc = 0.754460
Epoch 3
Validation auc = 0.752113
Epoch 4
Loss = 5.3151e-01, PNorm = 86.2015, GNorm = 11.9619, lr_0 = 6.5000e-04
Validation auc = 0.769484
Epoch 5
Validation auc = 0.718310
Epoch 6
Validation auc = 0.709859
Epoch 7
Validation auc = 0.655399
Epoch 8
Validation auc = 0.664789
Epoch 9
Loss = 3.8533e-01, PNorm = 86.5039, GNorm = 8.4978, lr_0 = 9.9743e-04
Validation auc = 0.676526
Epoch 10
Validation auc = 0.663380
Epoch 11
Validation auc = 0.627700
Epoch 12
Validation auc = 0.637089
Epoch 13
Validation auc = 0.671362
Epoch 14
Loss = 4.0804e-01, PNorm = 86.7822, GNorm = 5.3928, lr_0 = 9.8890e-04
Validation auc = 0.673239
Epoch 15
Validation auc = 0.675117
Epoch 16
Validation auc = 0.672300
Epoch 17
Validation auc = 0.669953
Epoch 18
Validation auc = 0.660094
Epoch 19
Loss = 2.1074e-01, PNorm = 86.9569, GNorm = 6.5756, lr_0 = 9.8045e-04
Validation auc = 0.662911
Epoch 20
Validation auc = 0.693427
Epoch 21
Validation auc = 0.709859
Epoch 22
Validation auc = 0.706103
Epoch 23
Validation auc = 0.699061
Epoch 24
Loss = 8.4767e-02, PNorm = 87.0732, GNorm = 1.0249, lr_0 = 9.7207e-04
Validation auc = 0.683568
Epoch 25
Validation auc = 0.674648
Epoch 26
Validation auc = 0.653052
Epoch 27
Validation auc = 0.675587
Epoch 28
Validation auc = 0.682629
Epoch 29
Loss = 1.0354e-01, PNorm = 87.1592, GNorm = 2.7741, lr_0 = 9.6376e-04
Validation auc = 0.686385
Epoch 30
Validation auc = 0.682629
Epoch 31
Validation auc = 0.679343
Epoch 32
Validation auc = 0.673239
Epoch 33
Validation auc = 0.661972
Epoch 34
Loss = 1.0842e-01, PNorm = 87.2559, GNorm = 1.3715, lr_0 = 9.5552e-04
Validation auc = 0.652582
Epoch 35
Validation auc = 0.654930
Epoch 36
Validation auc = 0.665728
Epoch 37
Validation auc = 0.682629
Epoch 38
Validation auc = 0.682160
Epoch 39
Loss = 2.4985e-01, PNorm = 87.3475, GNorm = 2.2534, lr_0 = 9.4735e-04
Validation auc = 0.684977
Epoch 40
Validation auc = 0.690141
Epoch 41
Validation auc = 0.706573
Epoch 42
Validation auc = 0.703756
Epoch 43
Validation auc = 0.704225
Epoch 44
Loss = 3.0645e-01, PNorm = 87.4825, GNorm = 7.4013, lr_0 = 9.3925e-04
Validation auc = 0.689671
Epoch 45
Validation auc = 0.682160
Epoch 46
Validation auc = 0.670423
Epoch 47
Validation auc = 0.666197
Epoch 48
Validation auc = 0.661972
Epoch 49
Loss = 8.7792e-02, PNorm = 87.6008, GNorm = 0.6895, lr_0 = 9.3122e-04
Validation auc = 0.668075
Epoch 50
Validation auc = 0.664789
Epoch 51
Validation auc = 0.663380
Epoch 52
Validation auc = 0.658216
Epoch 53
Validation auc = 0.658685
Epoch 54
Loss = 9.9569e-02, PNorm = 87.7221, GNorm = 3.5655, lr_0 = 9.2326e-04
Validation auc = 0.662441
Epoch 55
Validation auc = 0.661972
Epoch 56
Validation auc = 0.660563
Epoch 57
Validation auc = 0.656808
Epoch 58
Validation auc = 0.658216
Epoch 59
Loss = 2.1458e-02, PNorm = 87.8402, GNorm = 0.4921, lr_0 = 9.1537e-04
Validation auc = 0.658685
Epoch 60
Validation auc = 0.663380
Epoch 61
Validation auc = 0.668545
Epoch 62
Validation auc = 0.666667
Epoch 63
Validation auc = 0.676526
Epoch 64
Loss = 9.2756e-02, PNorm = 87.8984, GNorm = 2.5094, lr_0 = 9.0754e-04
Validation auc = 0.676526
Epoch 65
Validation auc = 0.681690
Epoch 66
Validation auc = 0.679812
Epoch 67
Validation auc = 0.675587
Epoch 68
Validation auc = 0.670892
Epoch 69
Loss = 2.4327e-02, PNorm = 87.9480, GNorm = 1.3212, lr_0 = 8.9978e-04
Validation auc = 0.666667
Epoch 70
Validation auc = 0.666667
Epoch 71
Validation auc = 0.663850
Epoch 72
Validation auc = 0.667606
Epoch 73
Validation auc = 0.672300
Epoch 74
Loss = 2.2813e-02, PNorm = 87.9921, GNorm = 0.8020, lr_0 = 8.9209e-04
Validation auc = 0.677934
Epoch 75
Validation auc = 0.682160
Epoch 76
Validation auc = 0.685446
Epoch 77
Validation auc = 0.683099
Epoch 78
Validation auc = 0.681690
Epoch 79
Loss = 3.4708e-02, PNorm = 88.0393, GNorm = 0.6623, lr_0 = 8.8447e-04
Validation auc = 0.682629
Epoch 80
Validation auc = 0.669484
Epoch 81
Validation auc = 0.663380
Epoch 82
Validation auc = 0.662911
Epoch 83
Validation auc = 0.662911
Epoch 84
Loss = 1.8370e-02, PNorm = 88.0861, GNorm = 1.4828, lr_0 = 8.7691e-04
Validation auc = 0.659624
Epoch 85
Validation auc = 0.656808
Epoch 86
Validation auc = 0.661502
Epoch 87
Validation auc = 0.672300
Epoch 88
Validation auc = 0.681221
Epoch 89
Loss = 7.5509e-02, PNorm = 88.1321, GNorm = 3.5262, lr_0 = 8.6941e-04
Validation auc = 0.694366
Epoch 90
Validation auc = 0.697183
Epoch 91
Validation auc = 0.700939
Epoch 92
Validation auc = 0.703756
Epoch 93
Validation auc = 0.700469
Epoch 94
Loss = 6.4162e-02, PNorm = 88.1721, GNorm = 2.5800, lr_0 = 8.6198e-04
Validation auc = 0.689671
Epoch 95
Validation auc = 0.673709
Epoch 96
Validation auc = 0.661502
Epoch 97
Validation auc = 0.650704
Epoch 98
Validation auc = 0.655869
Epoch 99
Loss = 1.8809e-02, PNorm = 88.2193, GNorm = 1.7045, lr_0 = 8.5461e-04
Validation auc = 0.667606
Epoch 100
Validation auc = 0.677934
Epoch 101
Validation auc = 0.684038
Epoch 102
Validation auc = 0.680282
Epoch 103
Validation auc = 0.673239
Epoch 104
Loss = 1.8893e-02, PNorm = 88.2725, GNorm = 1.3841, lr_0 = 8.4730e-04
Validation auc = 0.673709
Epoch 105
Validation auc = 0.680282
Epoch 106
Validation auc = 0.684507
Epoch 107
Validation auc = 0.690610
Epoch 108
Validation auc = 0.701408
Epoch 109
Loss = 1.5305e-02, PNorm = 88.3249, GNorm = 1.0729, lr_0 = 8.4006e-04
Validation auc = 0.706103
Epoch 110
Validation auc = 0.714085
Epoch 111
Validation auc = 0.712207
Epoch 112
Validation auc = 0.703286
Epoch 113
Validation auc = 0.695775
Epoch 114
Loss = 5.3807e-03, PNorm = 88.3954, GNorm = 0.1015, lr_0 = 8.3288e-04
Validation auc = 0.676056
Epoch 115
Validation auc = 0.668075
Epoch 116
Validation auc = 0.666197
Epoch 117
Validation auc = 0.664319
Epoch 118
Validation auc = 0.670892
Epoch 119
Loss = 2.9843e-02, PNorm = 88.4492, GNorm = 2.6529, lr_0 = 8.2576e-04
Validation auc = 0.674178
Epoch 120
Validation auc = 0.678404
Epoch 121
Validation auc = 0.684038
Epoch 122
Validation auc = 0.680282
Epoch 123
Validation auc = 0.679812
Epoch 124
Loss = 3.3310e-02, PNorm = 88.5866, GNorm = 2.7950, lr_0 = 8.1870e-04
Validation auc = 0.678404
Epoch 125
Validation auc = 0.674648
Epoch 126
Validation auc = 0.671831
Epoch 127
Validation auc = 0.664789
Epoch 128
Validation auc = 0.656338
Epoch 129
Loss = 9.5329e-03, PNorm = 88.7262, GNorm = 0.2435, lr_0 = 8.1170e-04
Validation auc = 0.660094
Epoch 130
Validation auc = 0.665258
Epoch 131
Validation auc = 0.677934
Epoch 132
Validation auc = 0.689671
Epoch 133
Validation auc = 0.692488
Epoch 134
Loss = 7.1493e-02, PNorm = 88.9239, GNorm = 2.0595, lr_0 = 8.0476e-04
Validation auc = 0.692019
Epoch 135
Validation auc = 0.692019
Epoch 136
Validation auc = 0.700469
Epoch 137
Validation auc = 0.700469
Epoch 138
Validation auc = 0.689202
Epoch 139
Loss = 6.1832e-02, PNorm = 89.1352, GNorm = 3.1139, lr_0 = 7.9788e-04
Validation auc = 0.688263
Epoch 140
Validation auc = 0.697653
Epoch 141
Validation auc = 0.709390
Epoch 142
Validation auc = 0.716901
Epoch 143
Validation auc = 0.711737
Epoch 144
Loss = 4.0663e-02, PNorm = 89.2729, GNorm = 2.2392, lr_0 = 7.9106e-04
Validation auc = 0.717840
Epoch 145
Validation auc = 0.701408
Epoch 146
Validation auc = 0.686385
Epoch 147
Validation auc = 0.677465
Epoch 148
Validation auc = 0.672300
Epoch 149
Loss = 1.1960e-02, PNorm = 89.3965, GNorm = 1.4540, lr_0 = 7.8430e-04
Validation auc = 0.669953
Epoch 150
Validation auc = 0.669014
Epoch 151
Validation auc = 0.670892
Epoch 152
Validation auc = 0.678404
Epoch 153
Validation auc = 0.683099
Epoch 154
Loss = 2.0636e-02, PNorm = 89.4895, GNorm = 0.1103, lr_0 = 7.7759e-04
Validation auc = 0.690141
Epoch 155
Validation auc = 0.684038
Epoch 156
Validation auc = 0.679343
Epoch 157
Validation auc = 0.679343
Epoch 158
Validation auc = 0.687793
Epoch 159
Loss = 1.8142e-02, PNorm = 89.5983, GNorm = 0.8548, lr_0 = 7.7095e-04
Validation auc = 0.697183
Epoch 160
Validation auc = 0.698122
Epoch 161
Validation auc = 0.701878
Epoch 162
Validation auc = 0.700469
Epoch 163
Validation auc = 0.707042
Epoch 164
Loss = 2.9352e-02, PNorm = 89.6824, GNorm = 1.3802, lr_0 = 7.6436e-04
Validation auc = 0.709859
Epoch 165
Validation auc = 0.708451
Epoch 166
Validation auc = 0.702817
Epoch 167
Validation auc = 0.703756
Epoch 168
Validation auc = 0.707042
Epoch 169
Loss = 4.9462e-03, PNorm = 89.7582, GNorm = 0.1659, lr_0 = 7.5782e-04
Validation auc = 0.704225
Epoch 170
Validation auc = 0.700939
Epoch 171
Validation auc = 0.699531
Epoch 172
Validation auc = 0.701408
Epoch 173
Validation auc = 0.700000
Epoch 174
Loss = 1.0982e-02, PNorm = 89.8115, GNorm = 0.9407, lr_0 = 7.5134e-04
Validation auc = 0.698122
Epoch 175
Validation auc = 0.695775
Epoch 176
Validation auc = 0.696244
Epoch 177
Validation auc = 0.697183
Epoch 178
Validation auc = 0.698122
Epoch 179
Loss = 2.6980e-03, PNorm = 89.8707, GNorm = 0.1692, lr_0 = 7.4492e-04
Validation auc = 0.699061
Epoch 180
Validation auc = 0.701408
Epoch 181
Validation auc = 0.701408
Epoch 182
Validation auc = 0.700469
Epoch 183
Validation auc = 0.697653
Epoch 184
Loss = 3.3445e-02, PNorm = 89.9084, GNorm = 1.4273, lr_0 = 7.3855e-04
Validation auc = 0.694366
Epoch 185
Validation auc = 0.688263
Epoch 186
Validation auc = 0.684038
Epoch 187
Validation auc = 0.681690
Epoch 188
Validation auc = 0.690610
Epoch 189
Loss = 4.3525e-03, PNorm = 89.9660, GNorm = 0.2426, lr_0 = 7.3224e-04
Validation auc = 0.694836
Epoch 190
Validation auc = 0.696714
Epoch 191
Validation auc = 0.700939
Epoch 192
Validation auc = 0.705634
Epoch 193
Validation auc = 0.705634
Epoch 194
Loss = 6.8992e-03, PNorm = 90.0273, GNorm = 0.4297, lr_0 = 7.2598e-04
Validation auc = 0.705164
Epoch 195
Validation auc = 0.699061
Epoch 196
Validation auc = 0.698592
Epoch 197
Validation auc = 0.698122
Epoch 198
Validation auc = 0.697653
Epoch 199
Loss = 3.4819e-03, PNorm = 90.1063, GNorm = 0.0865, lr_0 = 7.1977e-04
Validation auc = 0.698592
Epoch 200
Validation auc = 0.698592
Epoch 201
Validation auc = 0.700000
Epoch 202
Validation auc = 0.696714
Epoch 203
Validation auc = 0.696244
Epoch 204
Loss = 4.5432e-02, PNorm = 90.1568, GNorm = 3.4951, lr_0 = 7.1362e-04
Validation auc = 0.692019
Epoch 205
Validation auc = 0.697183
Epoch 206
Validation auc = 0.707981
Epoch 207
Validation auc = 0.723005
Epoch 208
Validation auc = 0.725822
Epoch 209
Loss = 1.0088e-01, PNorm = 90.2411, GNorm = 1.3862, lr_0 = 7.0752e-04
Validation auc = 0.716901
Epoch 210
Validation auc = 0.718310
Epoch 211
Validation auc = 0.716901
Epoch 212
Validation auc = 0.714085
Epoch 213
Validation auc = 0.714085
Epoch 214
Loss = 5.1953e-02, PNorm = 90.3396, GNorm = 0.2502, lr_0 = 7.0147e-04
Validation auc = 0.715023
Epoch 215
Validation auc = 0.713146
Epoch 216
Validation auc = 0.710798
Epoch 217
Validation auc = 0.710329
Epoch 218
Validation auc = 0.708920
Epoch 219
Loss = 3.0908e-03, PNorm = 90.4156, GNorm = 0.0137, lr_0 = 6.9548e-04
Validation auc = 0.705634
Epoch 220
Validation auc = 0.703286
Epoch 221
Validation auc = 0.699531
Epoch 222
Validation auc = 0.700000
Epoch 223
Validation auc = 0.700939
Epoch 224
Loss = 1.5391e-02, PNorm = 90.4549, GNorm = 0.0470, lr_0 = 6.8953e-04
Validation auc = 0.698122
Epoch 225
Validation auc = 0.692019
Epoch 226
Validation auc = 0.684977
Epoch 227
Validation auc = 0.689202
Epoch 228
Validation auc = 0.695305
Epoch 229
Loss = 2.9043e-03, PNorm = 90.5093, GNorm = 0.1073, lr_0 = 6.8364e-04
Validation auc = 0.695305
Epoch 230
Validation auc = 0.698122
Epoch 231
Validation auc = 0.701878
Epoch 232
Validation auc = 0.700939
Epoch 233
Validation auc = 0.709859
Epoch 234
Loss = 2.6716e-03, PNorm = 90.5713, GNorm = 0.2809, lr_0 = 6.7779e-04
Validation auc = 0.712207
Epoch 235
Validation auc = 0.712676
Epoch 236
Validation auc = 0.711268
Epoch 237
Validation auc = 0.715962
Epoch 238
Validation auc = 0.720188
Epoch 239
Loss = 3.1836e-02, PNorm = 90.6952, GNorm = 1.7635, lr_0 = 6.7200e-04
Validation auc = 0.723474
Epoch 240
Validation auc = 0.717840
Epoch 241
Validation auc = 0.710329
Epoch 242
Validation auc = 0.705634
Epoch 243
Validation auc = 0.702347
Epoch 244
Loss = 3.1060e-02, PNorm = 90.9665, GNorm = 0.4432, lr_0 = 6.6625e-04
Validation auc = 0.704695
Epoch 245
Validation auc = 0.706103
Epoch 246
Validation auc = 0.704225
Epoch 247
Validation auc = 0.710798
Epoch 248
Validation auc = 0.708920
Epoch 249
Loss = 1.7187e-02, PNorm = 91.1449, GNorm = 0.6983, lr_0 = 6.6056e-04
Validation auc = 0.694836
Epoch 250
Validation auc = 0.683099
Epoch 251
Validation auc = 0.682629
Epoch 252
Validation auc = 0.684038
Epoch 253
Validation auc = 0.679343
Epoch 254
Loss = 3.5473e-03, PNorm = 91.2777, GNorm = 0.2954, lr_0 = 6.5491e-04
Validation auc = 0.677934
Epoch 255
Validation auc = 0.675117
Epoch 256
Validation auc = 0.671362
Epoch 257
Validation auc = 0.673709
Epoch 258
Validation auc = 0.676056
Epoch 259
Loss = 1.2987e-01, PNorm = 91.3578, GNorm = 2.4860, lr_0 = 6.4931e-04
Validation auc = 0.684507
Epoch 260
Validation auc = 0.685915
Epoch 261
Validation auc = 0.690610
Epoch 262
Validation auc = 0.696244
Epoch 263
Validation auc = 0.705164
Epoch 264
Loss = 2.0597e-03, PNorm = 91.4286, GNorm = 0.0231, lr_0 = 6.4376e-04
Validation auc = 0.720188
Epoch 265
Validation auc = 0.732394
Epoch 266
Validation auc = 0.736150
Epoch 267
Validation auc = 0.735211
Epoch 268
Validation auc = 0.736620
Epoch 269
Loss = 1.6691e-03, PNorm = 91.4876, GNorm = 0.1352, lr_0 = 6.3826e-04
Validation auc = 0.735211
Epoch 270
Validation auc = 0.736150
Epoch 271
Validation auc = 0.729577
Epoch 272
Validation auc = 0.722535
Epoch 273
Validation auc = 0.712676
Epoch 274
Loss = 6.1341e-03, PNorm = 91.5321, GNorm = 0.5348, lr_0 = 6.3280e-04
Validation auc = 0.707981
Epoch 275
Validation auc = 0.697653
Epoch 276
Validation auc = 0.694836
Epoch 277
Validation auc = 0.690610
Epoch 278
Validation auc = 0.690141
Epoch 279
Loss = 8.9006e-03, PNorm = 91.5575, GNorm = 0.6629, lr_0 = 6.2739e-04
Validation auc = 0.688732
Epoch 280
Validation auc = 0.685915
Epoch 281
Validation auc = 0.691549
Epoch 282
Validation auc = 0.693897
Epoch 283
Validation auc = 0.696244
Epoch 284
Loss = 9.6496e-04, PNorm = 91.5917, GNorm = 0.0284, lr_0 = 6.2203e-04
Validation auc = 0.697653
Epoch 285
Validation auc = 0.699061
Epoch 286
Validation auc = 0.701408
Epoch 287
Validation auc = 0.700939
Epoch 288
Validation auc = 0.701878
Epoch 289
Loss = 1.3561e-03, PNorm = 91.6078, GNorm = 0.0202, lr_0 = 6.1671e-04
Validation auc = 0.701408
Epoch 290
Validation auc = 0.702347
Epoch 291
Validation auc = 0.699061
Epoch 292
Validation auc = 0.699531
Epoch 293
Validation auc = 0.701878
Epoch 294
Loss = 2.2311e-02, PNorm = 91.6381, GNorm = 2.7755, lr_0 = 6.1144e-04
Validation auc = 0.696244
Epoch 295
Validation auc = 0.696714
Epoch 296
Validation auc = 0.693427
Epoch 297
Validation auc = 0.690610
Epoch 298
Validation auc = 0.693427
Epoch 299
Loss = 1.5241e-02, PNorm = 91.6913, GNorm = 2.2909, lr_0 = 6.0621e-04
Validation auc = 0.693427
Model 0 best validation auc = 0.777465 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.833233
Ensemble test auc = 0.833233
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.708920
Epoch 1
Validation auc = 0.778169
Epoch 2
Validation auc = 0.767606
Epoch 3
Validation auc = 0.796714
Epoch 4
Loss = 5.7751e-01, PNorm = 86.2073, GNorm = 16.8339, lr_0 = 6.5000e-04
Validation auc = 0.763380
Epoch 5
Validation auc = 0.722066
Epoch 6
Validation auc = 0.707042
Epoch 7
Validation auc = 0.711737
Epoch 8
Validation auc = 0.720657
Epoch 9
Loss = 3.6598e-01, PNorm = 86.5111, GNorm = 6.9534, lr_0 = 9.9743e-04
Validation auc = 0.714554
Epoch 10
Validation auc = 0.659155
Epoch 11
Validation auc = 0.716901
Epoch 12
Validation auc = 0.744601
Epoch 13
Validation auc = 0.737089
Epoch 14
Loss = 3.2626e-01, PNorm = 86.7998, GNorm = 9.2470, lr_0 = 9.8890e-04
Validation auc = 0.721596
Epoch 15
Validation auc = 0.688732
Epoch 16
Validation auc = 0.690610
Epoch 17
Validation auc = 0.722535
Epoch 18
Validation auc = 0.756338
Epoch 19
Loss = 1.9584e-01, PNorm = 86.9922, GNorm = 7.4607, lr_0 = 9.8045e-04
Validation auc = 0.777934
Epoch 20
Validation auc = 0.764319
Epoch 21
Validation auc = 0.757277
Epoch 22
Validation auc = 0.738967
Epoch 23
Validation auc = 0.711268
Epoch 24
Loss = 2.1970e-01, PNorm = 87.1260, GNorm = 4.6473, lr_0 = 9.7207e-04
Validation auc = 0.700469
Epoch 25
Validation auc = 0.701878
Epoch 26
Validation auc = 0.727700
Epoch 27
Validation auc = 0.730986
Epoch 28
Validation auc = 0.741315
Epoch 29
Loss = 2.7204e-01, PNorm = 87.2247, GNorm = 2.8865, lr_0 = 9.6376e-04
Validation auc = 0.730516
Epoch 30
Validation auc = 0.702347
Epoch 31
Validation auc = 0.712676
Epoch 32
Validation auc = 0.724413
Epoch 33
Validation auc = 0.729108
Epoch 34
Loss = 7.9843e-02, PNorm = 87.3154, GNorm = 2.4147, lr_0 = 9.5552e-04
Validation auc = 0.727230
Epoch 35
Validation auc = 0.735681
Epoch 36
Validation auc = 0.737089
Epoch 37
Validation auc = 0.727700
Epoch 38
Validation auc = 0.723005
Epoch 39
Loss = 8.0156e-02, PNorm = 87.4037, GNorm = 1.2928, lr_0 = 9.4735e-04
Validation auc = 0.710798
Epoch 40
Validation auc = 0.696714
Epoch 41
Validation auc = 0.676056
Epoch 42
Validation auc = 0.651174
Epoch 43
Validation auc = 0.660094
Epoch 44
Loss = 9.8617e-02, PNorm = 87.4822, GNorm = 3.9800, lr_0 = 9.3925e-04
Validation auc = 0.684038
Epoch 45
Validation auc = 0.704695
Epoch 46
Validation auc = 0.715023
Epoch 47
Validation auc = 0.717840
Epoch 48
Validation auc = 0.719718
Epoch 49
Loss = 1.2581e-01, PNorm = 87.5597, GNorm = 2.1004, lr_0 = 9.3122e-04
Validation auc = 0.716432
Epoch 50
Validation auc = 0.711268
Epoch 51
Validation auc = 0.698592
Epoch 52
Validation auc = 0.707512
Epoch 53
Validation auc = 0.718310
Epoch 54
Loss = 3.2594e-02, PNorm = 87.6394, GNorm = 0.7803, lr_0 = 9.2326e-04
Validation auc = 0.725352
Epoch 55
Validation auc = 0.729108
Epoch 56
Validation auc = 0.724413
Epoch 57
Validation auc = 0.718779
Epoch 58
Validation auc = 0.710798
Epoch 59
Loss = 8.8539e-02, PNorm = 87.7139, GNorm = 2.4453, lr_0 = 9.1537e-04
Validation auc = 0.700469
Epoch 60
Validation auc = 0.701408
Epoch 61
Validation auc = 0.701408
Epoch 62
Validation auc = 0.676995
Epoch 63
Validation auc = 0.685446
Epoch 64
Loss = 5.2910e-02, PNorm = 87.7858, GNorm = 0.9662, lr_0 = 9.0754e-04
Validation auc = 0.694836
Epoch 65
Validation auc = 0.693427
Epoch 66
Validation auc = 0.692019
Epoch 67
Validation auc = 0.700939
Epoch 68
Validation auc = 0.706573
Epoch 69
Loss = 2.7411e-02, PNorm = 87.8671, GNorm = 0.5455, lr_0 = 8.9978e-04
Validation auc = 0.698592
Epoch 70
Validation auc = 0.683568
Epoch 71
Validation auc = 0.664789
Epoch 72
Validation auc = 0.655869
Epoch 73
Validation auc = 0.668075
Epoch 74
Loss = 2.9584e-02, PNorm = 88.0065, GNorm = 0.3139, lr_0 = 8.9209e-04
Validation auc = 0.678404
Epoch 75
Validation auc = 0.689671
Epoch 76
Validation auc = 0.689202
Epoch 77
Validation auc = 0.687324
Epoch 78
Validation auc = 0.675117
Epoch 79
Loss = 1.0888e-01, PNorm = 88.1038, GNorm = 6.3494, lr_0 = 8.8447e-04
Validation auc = 0.663850
Epoch 80
Validation auc = 0.665258
Epoch 81
Validation auc = 0.669953
Epoch 82
Validation auc = 0.684038
Epoch 83
Validation auc = 0.702347
Epoch 84
Loss = 5.1221e-02, PNorm = 88.2024, GNorm = 1.7857, lr_0 = 8.7691e-04
Validation auc = 0.710329
Epoch 85
Validation auc = 0.709859
Epoch 86
Validation auc = 0.702817
Epoch 87
Validation auc = 0.705634
Epoch 88
Validation auc = 0.705164
Epoch 89
Loss = 9.8269e-02, PNorm = 88.3141, GNorm = 1.5032, lr_0 = 8.6941e-04
Validation auc = 0.697653
Epoch 90
Validation auc = 0.688263
Epoch 91
Validation auc = 0.684977
Epoch 92
Validation auc = 0.677934
Epoch 93
Validation auc = 0.672300
Epoch 94
Loss = 3.6991e-02, PNorm = 88.4109, GNorm = 0.5943, lr_0 = 8.6198e-04
Validation auc = 0.662911
Epoch 95
Validation auc = 0.656808
Epoch 96
Validation auc = 0.655869
Epoch 97
Validation auc = 0.678404
Epoch 98
Validation auc = 0.714085
Epoch 99
Loss = 3.0851e-02, PNorm = 88.4941, GNorm = 1.7027, lr_0 = 8.5461e-04
Validation auc = 0.720657
Epoch 100
Validation auc = 0.714085
Epoch 101
Validation auc = 0.695775
Epoch 102
Validation auc = 0.684038
Epoch 103
Validation auc = 0.683099
Epoch 104
Loss = 1.7215e-02, PNorm = 88.5581, GNorm = 0.6888, lr_0 = 8.4730e-04
Validation auc = 0.675587
Epoch 105
Validation auc = 0.672770
Epoch 106
Validation auc = 0.674648
Epoch 107
Validation auc = 0.674178
Epoch 108
Validation auc = 0.671831
Epoch 109
Loss = 5.0293e-02, PNorm = 88.6055, GNorm = 2.3760, lr_0 = 8.4006e-04
Validation auc = 0.670423
Epoch 110
Validation auc = 0.673709
Epoch 111
Validation auc = 0.660094
Epoch 112
Validation auc = 0.661972
Epoch 113
Validation auc = 0.660563
Epoch 114
Loss = 2.2513e-02, PNorm = 88.6605, GNorm = 1.3474, lr_0 = 8.3288e-04
Validation auc = 0.658216
Epoch 115
Validation auc = 0.661502
Epoch 116
Validation auc = 0.665258
Epoch 117
Validation auc = 0.659624
Epoch 118
Validation auc = 0.678404
Epoch 119
Loss = 3.4883e-02, PNorm = 88.7332, GNorm = 1.1977, lr_0 = 8.2576e-04
Validation auc = 0.688263
Epoch 120
Validation auc = 0.703286
Epoch 121
Validation auc = 0.711268
Epoch 122
Validation auc = 0.713146
Epoch 123
Validation auc = 0.707512
Epoch 124
Loss = 1.3877e-01, PNorm = 88.7902, GNorm = 4.1340, lr_0 = 8.1870e-04
Validation auc = 0.697183
Epoch 125
Validation auc = 0.691080
Epoch 126
Validation auc = 0.670423
Epoch 127
Validation auc = 0.661502
Epoch 128
Validation auc = 0.655869
Epoch 129
Loss = 3.5339e-02, PNorm = 88.8448, GNorm = 2.0288, lr_0 = 8.1170e-04
Validation auc = 0.668075
Epoch 130
Validation auc = 0.681690
Epoch 131
Validation auc = 0.690141
Epoch 132
Validation auc = 0.697183
Epoch 133
Validation auc = 0.700000
Epoch 134
Loss = 2.3531e-02, PNorm = 88.8906, GNorm = 1.0965, lr_0 = 8.0476e-04
Validation auc = 0.708920
Epoch 135
Validation auc = 0.709859
Epoch 136
Validation auc = 0.708451
Epoch 137
Validation auc = 0.704695
Epoch 138
Validation auc = 0.701878
Epoch 139
Loss = 6.4545e-02, PNorm = 88.9288, GNorm = 3.3952, lr_0 = 7.9788e-04
Validation auc = 0.694836
Epoch 140
Validation auc = 0.697653
Epoch 141
Validation auc = 0.699061
Epoch 142
Validation auc = 0.707981
Epoch 143
Validation auc = 0.702817
Epoch 144
Loss = 4.9234e-02, PNorm = 88.9711, GNorm = 1.7193, lr_0 = 7.9106e-04
Validation auc = 0.690141
Epoch 145
Validation auc = 0.685446
Epoch 146
Validation auc = 0.681221
Epoch 147
Validation auc = 0.684977
Epoch 148
Validation auc = 0.689202
Epoch 149
Loss = 4.3976e-02, PNorm = 89.0119, GNorm = 1.4969, lr_0 = 7.8430e-04
Validation auc = 0.677465
Epoch 150
Validation auc = 0.676526
Epoch 151
Validation auc = 0.679343
Epoch 152
Validation auc = 0.684038
Epoch 153
Validation auc = 0.685446
Epoch 154
Loss = 1.4537e-02, PNorm = 89.0551, GNorm = 0.6069, lr_0 = 7.7759e-04
Validation auc = 0.683568
Epoch 155
Validation auc = 0.680282
Epoch 156
Validation auc = 0.674178
Epoch 157
Validation auc = 0.672770
Epoch 158
Validation auc = 0.674648
Epoch 159
Loss = 1.3368e-02, PNorm = 89.0935, GNorm = 0.4469, lr_0 = 7.7095e-04
Validation auc = 0.675587
Epoch 160
Validation auc = 0.680751
Epoch 161
Validation auc = 0.678873
Epoch 162
Validation auc = 0.676995
Epoch 163
Validation auc = 0.673239
Epoch 164
Loss = 4.0758e-03, PNorm = 89.1250, GNorm = 0.1551, lr_0 = 7.6436e-04
Validation auc = 0.662911
Epoch 165
Validation auc = 0.658216
Epoch 166
Validation auc = 0.654930
Epoch 167
Validation auc = 0.654460
Epoch 168
Validation auc = 0.653052
Epoch 169
Loss = 2.9198e-02, PNorm = 89.1551, GNorm = 0.1542, lr_0 = 7.5782e-04
Validation auc = 0.658216
Epoch 170
Validation auc = 0.661502
Epoch 171
Validation auc = 0.667606
Epoch 172
Validation auc = 0.676056
Epoch 173
Validation auc = 0.675117
Epoch 174
Loss = 4.9127e-02, PNorm = 89.2092, GNorm = 0.2310, lr_0 = 7.5134e-04
Validation auc = 0.676526
Epoch 175
Validation auc = 0.670423
Epoch 176
Validation auc = 0.659624
Epoch 177
Validation auc = 0.657277
Epoch 178
Validation auc = 0.656808
Epoch 179
Loss = 1.1252e-02, PNorm = 89.2687, GNorm = 1.3351, lr_0 = 7.4492e-04
Validation auc = 0.656808
Epoch 180
Validation auc = 0.656338
Epoch 181
Validation auc = 0.655399
Epoch 182
Validation auc = 0.655869
Epoch 183
Validation auc = 0.654930
Epoch 184
Loss = 1.5418e-02, PNorm = 89.3132, GNorm = 0.6089, lr_0 = 7.3855e-04
Validation auc = 0.659155
Epoch 185
Validation auc = 0.656338
Epoch 186
Validation auc = 0.656808
Epoch 187
Validation auc = 0.656808
Epoch 188
Validation auc = 0.662441
Epoch 189
Loss = 5.0754e-03, PNorm = 89.3381, GNorm = 0.3621, lr_0 = 7.3224e-04
Validation auc = 0.667606
Epoch 190
Validation auc = 0.669014
Epoch 191
Validation auc = 0.674648
Epoch 192
Validation auc = 0.675117
Epoch 193
Validation auc = 0.669484
Epoch 194
Loss = 2.0863e-02, PNorm = 89.3838, GNorm = 1.2518, lr_0 = 7.2598e-04
Validation auc = 0.668545
Epoch 195
Validation auc = 0.664789
Epoch 196
Validation auc = 0.658216
Epoch 197
Validation auc = 0.654460
Epoch 198
Validation auc = 0.653052
Epoch 199
Loss = 7.4865e-03, PNorm = 89.4659, GNorm = 0.4184, lr_0 = 7.1977e-04
Validation auc = 0.650235
Epoch 200
Validation auc = 0.648357
Epoch 201
Validation auc = 0.647418
Epoch 202
Validation auc = 0.645540
Epoch 203
Validation auc = 0.641315
Epoch 204
Loss = 5.5906e-03, PNorm = 89.5115, GNorm = 0.6524, lr_0 = 7.1362e-04
Validation auc = 0.641315
Epoch 205
Validation auc = 0.644131
Epoch 206
Validation auc = 0.648826
Epoch 207
Validation auc = 0.652582
Epoch 208
Validation auc = 0.658685
Epoch 209
Loss = 1.3717e-02, PNorm = 89.5862, GNorm = 0.4611, lr_0 = 7.0752e-04
Validation auc = 0.661972
Epoch 210
Validation auc = 0.663380
Epoch 211
Validation auc = 0.669484
Epoch 212
Validation auc = 0.670892
Epoch 213
Validation auc = 0.674178
Epoch 214
Loss = 1.9222e-02, PNorm = 89.6715, GNorm = 0.2948, lr_0 = 7.0147e-04
Validation auc = 0.683099
Epoch 215
Validation auc = 0.682629
Epoch 216
Validation auc = 0.676056
Epoch 217
Validation auc = 0.669014
Epoch 218
Validation auc = 0.661502
Epoch 219
Loss = 3.2331e-03, PNorm = 89.7534, GNorm = 0.0284, lr_0 = 6.9548e-04
Validation auc = 0.655869
Epoch 220
Validation auc = 0.648826
Epoch 221
Validation auc = 0.642254
Epoch 222
Validation auc = 0.645070
Epoch 223
Validation auc = 0.645070
Epoch 224
Loss = 2.9228e-03, PNorm = 89.8142, GNorm = 0.2270, lr_0 = 6.8953e-04
Validation auc = 0.651643
Epoch 225
Validation auc = 0.653991
Epoch 226
Validation auc = 0.656808
Epoch 227
Validation auc = 0.660094
Epoch 228
Validation auc = 0.666197
Epoch 229
Loss = 5.3270e-03, PNorm = 89.8542, GNorm = 0.4995, lr_0 = 6.8364e-04
Validation auc = 0.667606
Epoch 230
Validation auc = 0.668545
Epoch 231
Validation auc = 0.669953
Epoch 232
Validation auc = 0.662911
Epoch 233
Validation auc = 0.660094
Epoch 234
Loss = 2.5954e-03, PNorm = 89.8821, GNorm = 0.2737, lr_0 = 6.7779e-04
Validation auc = 0.659155
Epoch 235
Validation auc = 0.661972
Epoch 236
Validation auc = 0.663850
Epoch 237
Validation auc = 0.660563
Epoch 238
Validation auc = 0.654930
Epoch 239
Loss = 4.6589e-03, PNorm = 89.9208, GNorm = 0.4284, lr_0 = 6.7200e-04
Validation auc = 0.653052
Epoch 240
Validation auc = 0.649296
Epoch 241
Validation auc = 0.648357
Epoch 242
Validation auc = 0.648357
Epoch 243
Validation auc = 0.650235
Epoch 244
Loss = 2.6188e-02, PNorm = 89.9909, GNorm = 2.2001, lr_0 = 6.6625e-04
Validation auc = 0.655869
Epoch 245
Validation auc = 0.660563
Epoch 246
Validation auc = 0.664319
Epoch 247
Validation auc = 0.667606
Epoch 248
Validation auc = 0.671831
Epoch 249
Loss = 1.2666e-03, PNorm = 90.0506, GNorm = 0.0534, lr_0 = 6.6056e-04
Validation auc = 0.676056
Epoch 250
Validation auc = 0.677934
Epoch 251
Validation auc = 0.675587
Epoch 252
Validation auc = 0.667606
Epoch 253
Validation auc = 0.666667
Epoch 254
Loss = 2.9363e-02, PNorm = 90.0689, GNorm = 0.7691, lr_0 = 6.5491e-04
Validation auc = 0.668075
Epoch 255
Validation auc = 0.666197
Epoch 256
Validation auc = 0.666197
Epoch 257
Validation auc = 0.669014
Epoch 258
Validation auc = 0.669014
Epoch 259
Loss = 2.0195e-02, PNorm = 90.1069, GNorm = 1.5828, lr_0 = 6.4931e-04
Validation auc = 0.671831
Epoch 260
Validation auc = 0.682629
Epoch 261
Validation auc = 0.681690
Epoch 262
Validation auc = 0.684507
Epoch 263
Validation auc = 0.689671
Epoch 264
Loss = 9.9674e-04, PNorm = 90.1770, GNorm = 0.0336, lr_0 = 6.4376e-04
Validation auc = 0.687793
Epoch 265
Validation auc = 0.683099
Epoch 266
Validation auc = 0.690141
Epoch 267
Validation auc = 0.687793
Epoch 268
Validation auc = 0.687793
Epoch 269
Loss = 3.8444e-03, PNorm = 90.2409, GNorm = 0.7233, lr_0 = 6.3826e-04
Validation auc = 0.689671
Epoch 270
Validation auc = 0.689671
Epoch 271
Validation auc = 0.684038
Epoch 272
Validation auc = 0.680282
Epoch 273
Validation auc = 0.680282
Epoch 274
Loss = 1.0322e-02, PNorm = 90.2953, GNorm = 0.3031, lr_0 = 6.3280e-04
Validation auc = 0.682629
Epoch 275
Validation auc = 0.683568
Epoch 276
Validation auc = 0.686854
Epoch 277
Validation auc = 0.685915
Epoch 278
Validation auc = 0.687324
Epoch 279
Loss = 6.0523e-03, PNorm = 90.3364, GNorm = 1.2491, lr_0 = 6.2739e-04
Validation auc = 0.686854
Epoch 280
Validation auc = 0.680282
Epoch 281
Validation auc = 0.676995
Epoch 282
Validation auc = 0.677934
Epoch 283
Validation auc = 0.680282
Epoch 284
Loss = 3.1610e-03, PNorm = 90.3783, GNorm = 0.5744, lr_0 = 6.2203e-04
Validation auc = 0.682160
Epoch 285
Validation auc = 0.683099
Epoch 286
Validation auc = 0.684038
Epoch 287
Validation auc = 0.685446
Epoch 288
Validation auc = 0.689202
Epoch 289
Loss = 5.6153e-04, PNorm = 90.4088, GNorm = 0.0224, lr_0 = 6.1671e-04
Validation auc = 0.692488
Epoch 290
Validation auc = 0.693897
Epoch 291
Validation auc = 0.693897
Epoch 292
Validation auc = 0.696714
Epoch 293
Validation auc = 0.692488
Epoch 294
Loss = 3.9797e-03, PNorm = 90.4401, GNorm = 0.0999, lr_0 = 6.1144e-04
Validation auc = 0.690610
Epoch 295
Validation auc = 0.689202
Epoch 296
Validation auc = 0.686385
Epoch 297
Validation auc = 0.684038
Epoch 298
Validation auc = 0.683568
Epoch 299
Loss = 3.0628e-03, PNorm = 90.4746, GNorm = 0.1845, lr_0 = 6.0621e-04
Validation auc = 0.682160
Model 0 best validation auc = 0.796714 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.886213
Ensemble test auc = 0.886213
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.728169
Epoch 1
Validation auc = 0.716432
Epoch 2
Validation auc = 0.729577
Epoch 3
Validation auc = 0.734742
Epoch 4
Loss = 5.7047e-01, PNorm = 86.2065, GNorm = 6.2741, lr_0 = 6.5000e-04
Validation auc = 0.673709
Epoch 5
Validation auc = 0.626761
Epoch 6
Validation auc = 0.624413
Epoch 7
Validation auc = 0.663380
Epoch 8
Validation auc = 0.663850
Epoch 9
Loss = 4.0562e-01, PNorm = 86.5064, GNorm = 12.6150, lr_0 = 9.9743e-04
Validation auc = 0.625822
Epoch 10
Validation auc = 0.636150
Epoch 11
Validation auc = 0.659624
Epoch 12
Validation auc = 0.695775
Epoch 13
Validation auc = 0.693427
Epoch 14
Loss = 2.3938e-01, PNorm = 86.8058, GNorm = 3.2274, lr_0 = 9.8890e-04
Validation auc = 0.662441
Epoch 15
Validation auc = 0.632864
Epoch 16
Validation auc = 0.646009
Epoch 17
Validation auc = 0.667136
Epoch 18
Validation auc = 0.692488
Epoch 19
Loss = 1.0877e-01, PNorm = 87.0139, GNorm = 1.7436, lr_0 = 9.8045e-04
Validation auc = 0.706573
Epoch 20
Validation auc = 0.708920
Epoch 21
Validation auc = 0.692958
Epoch 22
Validation auc = 0.683568
Epoch 23
Validation auc = 0.687324
Epoch 24
Loss = 1.0602e-01, PNorm = 87.1736, GNorm = 5.0528, lr_0 = 9.7207e-04
Validation auc = 0.673239
Epoch 25
Validation auc = 0.650704
Epoch 26
Validation auc = 0.649296
Epoch 27
Validation auc = 0.654460
Epoch 28
Validation auc = 0.700469
Epoch 29
Loss = 1.7641e-01, PNorm = 87.3339, GNorm = 5.4699, lr_0 = 9.6376e-04
Validation auc = 0.705634
Epoch 30
Validation auc = 0.704225
Epoch 31
Validation auc = 0.703756
Epoch 32
Validation auc = 0.704695
Epoch 33
Validation auc = 0.707981
Epoch 34
Loss = 4.3307e-02, PNorm = 87.4734, GNorm = 0.7697, lr_0 = 9.5552e-04
Validation auc = 0.700000
Epoch 35
Validation auc = 0.698592
Epoch 36
Validation auc = 0.700000
Epoch 37
Validation auc = 0.715493
Epoch 38
Validation auc = 0.693897
Epoch 39
Loss = 1.0352e-01, PNorm = 87.5695, GNorm = 0.6285, lr_0 = 9.4735e-04
Validation auc = 0.696244
Epoch 40
Validation auc = 0.699061
Epoch 41
Validation auc = 0.686854
Epoch 42
Validation auc = 0.661972
Epoch 43
Validation auc = 0.652113
Epoch 44
Loss = 1.3625e-01, PNorm = 87.6788, GNorm = 1.6083, lr_0 = 9.3925e-04
Validation auc = 0.656808
Epoch 45
Validation auc = 0.663850
Epoch 46
Validation auc = 0.669484
Epoch 47
Validation auc = 0.680282
Epoch 48
Validation auc = 0.702347
Epoch 49
Loss = 1.8297e-01, PNorm = 87.7832, GNorm = 4.5169, lr_0 = 9.3122e-04
Validation auc = 0.698122
Epoch 50
Validation auc = 0.681690
Epoch 51
Validation auc = 0.664319
Epoch 52
Validation auc = 0.651174
Epoch 53
Validation auc = 0.650704
Epoch 54
Loss = 2.2522e-02, PNorm = 87.8560, GNorm = 1.4091, lr_0 = 9.2326e-04
Validation auc = 0.644131
Epoch 55
Validation auc = 0.635211
Epoch 56
Validation auc = 0.621127
Epoch 57
Validation auc = 0.620188
Epoch 58
Validation auc = 0.624883
Epoch 59
Loss = 3.3505e-02, PNorm = 87.9087, GNorm = 1.1056, lr_0 = 9.1537e-04
Validation auc = 0.631925
Epoch 60
Validation auc = 0.650704
Epoch 61
Validation auc = 0.656338
Epoch 62
Validation auc = 0.676995
Epoch 63
Validation auc = 0.702347
Epoch 64
Loss = 2.5472e-02, PNorm = 87.9607, GNorm = 0.3173, lr_0 = 9.0754e-04
Validation auc = 0.709390
Epoch 65
Validation auc = 0.689671
Epoch 66
Validation auc = 0.686854
Epoch 67
Validation auc = 0.692958
Epoch 68
Validation auc = 0.689202
Epoch 69
Loss = 1.7803e-01, PNorm = 88.1074, GNorm = 3.5335, lr_0 = 8.9978e-04
Validation auc = 0.673709
Epoch 70
Validation auc = 0.677934
Epoch 71
Validation auc = 0.677465
Epoch 72
Validation auc = 0.683568
Epoch 73
Validation auc = 0.691549
Epoch 74
Loss = 1.0320e-02, PNorm = 88.2339, GNorm = 0.5388, lr_0 = 8.9209e-04
Validation auc = 0.699061
Epoch 75
Validation auc = 0.687793
Epoch 76
Validation auc = 0.679812
Epoch 77
Validation auc = 0.674648
Epoch 78
Validation auc = 0.674648
Epoch 79
Loss = 3.7533e-02, PNorm = 88.3556, GNorm = 1.0408, lr_0 = 8.8447e-04
Validation auc = 0.677465
Epoch 80
Validation auc = 0.681221
Epoch 81
Validation auc = 0.686385
Epoch 82
Validation auc = 0.685446
Epoch 83
Validation auc = 0.676526
Epoch 84
Loss = 1.4402e-02, PNorm = 88.4421, GNorm = 0.4619, lr_0 = 8.7691e-04
Validation auc = 0.676056
Epoch 85
Validation auc = 0.667606
Epoch 86
Validation auc = 0.661033
Epoch 87
Validation auc = 0.670423
Epoch 88
Validation auc = 0.670423
Epoch 89
Loss = 3.4620e-02, PNorm = 88.4937, GNorm = 1.1540, lr_0 = 8.6941e-04
Validation auc = 0.661033
Epoch 90
Validation auc = 0.650235
Epoch 91
Validation auc = 0.646948
Epoch 92
Validation auc = 0.635211
Epoch 93
Validation auc = 0.628638
Epoch 94
Loss = 4.3805e-02, PNorm = 88.5365, GNorm = 2.5110, lr_0 = 8.6198e-04
Validation auc = 0.629577
Epoch 95
Validation auc = 0.631925
Epoch 96
Validation auc = 0.638028
Epoch 97
Validation auc = 0.638498
Epoch 98
Validation auc = 0.644131
Epoch 99
Loss = 3.8639e-02, PNorm = 88.5780, GNorm = 1.0239, lr_0 = 8.5461e-04
Validation auc = 0.650704
Epoch 100
Validation auc = 0.658685
Epoch 101
Validation auc = 0.661972
Epoch 102
Validation auc = 0.654460
Epoch 103
Validation auc = 0.643192
Epoch 104
Loss = 4.5276e-02, PNorm = 88.6257, GNorm = 1.7224, lr_0 = 8.4730e-04
Validation auc = 0.633803
Epoch 105
Validation auc = 0.626291
Epoch 106
Validation auc = 0.623474
Epoch 107
Validation auc = 0.633803
Epoch 108
Validation auc = 0.649765
Epoch 109
Loss = 4.3913e-02, PNorm = 88.6658, GNorm = 0.2840, lr_0 = 8.4006e-04
Validation auc = 0.655399
Epoch 110
Validation auc = 0.654930
Epoch 111
Validation auc = 0.663380
Epoch 112
Validation auc = 0.669014
Epoch 113
Validation auc = 0.674178
Epoch 114
Loss = 1.9157e-01, PNorm = 88.7051, GNorm = 2.7671, lr_0 = 8.3288e-04
Validation auc = 0.661972
Epoch 115
Validation auc = 0.643662
Epoch 116
Validation auc = 0.652582
Epoch 117
Validation auc = 0.663850
Epoch 118
Validation auc = 0.676056
Epoch 119
Loss = 1.0688e-01, PNorm = 88.7532, GNorm = 2.5440, lr_0 = 8.2576e-04
Validation auc = 0.688263
Epoch 120
Validation auc = 0.681690
Epoch 121
Validation auc = 0.660094
Epoch 122
Validation auc = 0.660563
Epoch 123
Validation auc = 0.647418
Epoch 124
Loss = 1.3537e-01, PNorm = 88.8153, GNorm = 2.5780, lr_0 = 8.1870e-04
Validation auc = 0.649765
Epoch 125
Validation auc = 0.643192
Epoch 126
Validation auc = 0.634272
Epoch 127
Validation auc = 0.629108
Epoch 128
Validation auc = 0.620657
Epoch 129
Loss = 1.7516e-01, PNorm = 88.8929, GNorm = 4.9929, lr_0 = 8.1170e-04
Validation auc = 0.645070
Epoch 130
Validation auc = 0.650235
Epoch 131
Validation auc = 0.649296
Epoch 132
Validation auc = 0.648357
Epoch 133
Validation auc = 0.636150
Epoch 134
Loss = 5.1007e-02, PNorm = 88.9687, GNorm = 2.0674, lr_0 = 8.0476e-04
Validation auc = 0.633803
Epoch 135
Validation auc = 0.631455
Epoch 136
Validation auc = 0.632864
Epoch 137
Validation auc = 0.648357
Epoch 138
Validation auc = 0.662441
Epoch 139
Loss = 5.7176e-02, PNorm = 89.0507, GNorm = 2.8324, lr_0 = 7.9788e-04
Validation auc = 0.682160
Epoch 140
Validation auc = 0.681690
Epoch 141
Validation auc = 0.686385
Epoch 142
Validation auc = 0.688263
Epoch 143
Validation auc = 0.683568
Epoch 144
Loss = 4.2104e-02, PNorm = 89.1534, GNorm = 3.0278, lr_0 = 7.9106e-04
Validation auc = 0.687324
Epoch 145
Validation auc = 0.667606
Epoch 146
Validation auc = 0.653052
Epoch 147
Validation auc = 0.649296
Epoch 148
Validation auc = 0.649765
Epoch 149
Loss = 1.1623e-02, PNorm = 89.2887, GNorm = 0.5364, lr_0 = 7.8430e-04
Validation auc = 0.647887
Epoch 150
Validation auc = 0.646009
Epoch 151
Validation auc = 0.660563
Epoch 152
Validation auc = 0.676056
Epoch 153
Validation auc = 0.682160
Epoch 154
Loss = 9.7218e-02, PNorm = 89.4072, GNorm = 5.8116, lr_0 = 7.7759e-04
Validation auc = 0.685915
Epoch 155
Validation auc = 0.683568
Epoch 156
Validation auc = 0.688732
Epoch 157
Validation auc = 0.682629
Epoch 158
Validation auc = 0.673709
Epoch 159
Loss = 8.9727e-02, PNorm = 89.4949, GNorm = 1.0674, lr_0 = 7.7095e-04
Validation auc = 0.669953
Epoch 160
Validation auc = 0.671362
Epoch 161
Validation auc = 0.668545
Epoch 162
Validation auc = 0.680751
Epoch 163
Validation auc = 0.677465
Epoch 164
Loss = 1.2422e-01, PNorm = 89.5939, GNorm = 2.3837, lr_0 = 7.6436e-04
Validation auc = 0.676056
Epoch 165
Validation auc = 0.671362
Epoch 166
Validation auc = 0.665728
Epoch 167
Validation auc = 0.657277
Epoch 168
Validation auc = 0.652113
Epoch 169
Loss = 2.0861e-02, PNorm = 89.6564, GNorm = 1.3527, lr_0 = 7.5782e-04
Validation auc = 0.644131
Epoch 170
Validation auc = 0.650704
Epoch 171
Validation auc = 0.652582
Epoch 172
Validation auc = 0.653521
Epoch 173
Validation auc = 0.662441
Epoch 174
Loss = 2.9414e-03, PNorm = 89.6992, GNorm = 0.1489, lr_0 = 7.5134e-04
Validation auc = 0.670423
Epoch 175
Validation auc = 0.675587
Epoch 176
Validation auc = 0.680751
Epoch 177
Validation auc = 0.681221
Epoch 178
Validation auc = 0.681221
Epoch 179
Loss = 1.2559e-02, PNorm = 89.7428, GNorm = 0.5962, lr_0 = 7.4492e-04
Validation auc = 0.676526
Epoch 180
Validation auc = 0.675587
Epoch 181
Validation auc = 0.680751
Epoch 182
Validation auc = 0.681221
Epoch 183
Validation auc = 0.681690
Epoch 184
Loss = 2.4001e-03, PNorm = 89.7721, GNorm = 0.0440, lr_0 = 7.3855e-04
Validation auc = 0.680751
Epoch 185
Validation auc = 0.681221
Epoch 186
Validation auc = 0.681221
Epoch 187
Validation auc = 0.682629
Epoch 188
Validation auc = 0.681690
Epoch 189
Loss = 6.0867e-03, PNorm = 89.7897, GNorm = 0.1203, lr_0 = 7.3224e-04
Validation auc = 0.679343
Epoch 190
Validation auc = 0.675117
Epoch 191
Validation auc = 0.672300
Epoch 192
Validation auc = 0.670423
Epoch 193
Validation auc = 0.668545
Epoch 194
Loss = 1.4165e-02, PNorm = 89.8066, GNorm = 0.3168, lr_0 = 7.2598e-04
Validation auc = 0.665728
Epoch 195
Validation auc = 0.667606
Epoch 196
Validation auc = 0.666667
Epoch 197
Validation auc = 0.669014
Epoch 198
Validation auc = 0.669953
Epoch 199
Loss = 8.4123e-03, PNorm = 89.8273, GNorm = 0.2577, lr_0 = 7.1977e-04
Validation auc = 0.674178
Epoch 200
Validation auc = 0.679812
Epoch 201
Validation auc = 0.693427
Epoch 202
Validation auc = 0.701408
Epoch 203
Validation auc = 0.710329
Epoch 204
Loss = 1.8253e-02, PNorm = 89.8517, GNorm = 2.0780, lr_0 = 7.1362e-04
Validation auc = 0.712207
Epoch 205
Validation auc = 0.701878
Epoch 206
Validation auc = 0.683099
Epoch 207
Validation auc = 0.670423
Epoch 208
Validation auc = 0.676056
Epoch 209
Loss = 8.5070e-03, PNorm = 89.8933, GNorm = 0.9971, lr_0 = 7.0752e-04
Validation auc = 0.684038
Epoch 210
Validation auc = 0.696714
Epoch 211
Validation auc = 0.704695
Epoch 212
Validation auc = 0.700469
Epoch 213
Validation auc = 0.695305
Epoch 214
Loss = 4.7619e-03, PNorm = 89.9307, GNorm = 0.2988, lr_0 = 7.0147e-04
Validation auc = 0.690610
Epoch 215
Validation auc = 0.683099
Epoch 216
Validation auc = 0.677465
Epoch 217
Validation auc = 0.673709
Epoch 218
Validation auc = 0.674648
Epoch 219
Loss = 1.1349e-02, PNorm = 89.9606, GNorm = 0.6646, lr_0 = 6.9548e-04
Validation auc = 0.673709
Epoch 220
Validation auc = 0.672770
Epoch 221
Validation auc = 0.671831
Epoch 222
Validation auc = 0.674648
Epoch 223
Validation auc = 0.682629
Epoch 224
Loss = 1.3067e-03, PNorm = 89.9888, GNorm = 0.0578, lr_0 = 6.8953e-04
Validation auc = 0.687324
Epoch 225
Validation auc = 0.688263
Epoch 226
Validation auc = 0.693427
Epoch 227
Validation auc = 0.694366
Epoch 228
Validation auc = 0.696714
Epoch 229
Loss = 8.5192e-03, PNorm = 90.0113, GNorm = 0.1373, lr_0 = 6.8364e-04
Validation auc = 0.689671
Epoch 230
Validation auc = 0.689202
Epoch 231
Validation auc = 0.684977
Epoch 232
Validation auc = 0.682160
Epoch 233
Validation auc = 0.677934
Epoch 234
Loss = 1.7889e-03, PNorm = 90.0374, GNorm = 0.0233, lr_0 = 6.7779e-04
Validation auc = 0.676526
Epoch 235
Validation auc = 0.678404
Epoch 236
Validation auc = 0.681690
Epoch 237
Validation auc = 0.684507
Epoch 238
Validation auc = 0.687324
Epoch 239
Loss = 3.6690e-02, PNorm = 90.0621, GNorm = 0.1600, lr_0 = 6.7200e-04
Validation auc = 0.685915
Epoch 240
Validation auc = 0.678404
Epoch 241
Validation auc = 0.674178
Epoch 242
Validation auc = 0.674648
Epoch 243
Validation auc = 0.674648
Epoch 244
Loss = 2.5052e-02, PNorm = 90.1069, GNorm = 3.3242, lr_0 = 6.6625e-04
Validation auc = 0.677934
Epoch 245
Validation auc = 0.676995
Epoch 246
Validation auc = 0.675587
Epoch 247
Validation auc = 0.677465
Epoch 248
Validation auc = 0.679343
Epoch 249
Loss = 8.7497e-04, PNorm = 90.1430, GNorm = 0.0387, lr_0 = 6.6056e-04
Validation auc = 0.679812
Epoch 250
Validation auc = 0.679812
Epoch 251
Validation auc = 0.681690
Epoch 252
Validation auc = 0.701878
Epoch 253
Validation auc = 0.720188
Epoch 254
Loss = 4.2400e-03, PNorm = 90.1735, GNorm = 0.4729, lr_0 = 6.5491e-04
Validation auc = 0.727230
Epoch 255
Validation auc = 0.735681
Epoch 256
Validation auc = 0.738498
Epoch 257
Validation auc = 0.725352
Epoch 258
Validation auc = 0.716901
Epoch 259
Loss = 5.5020e-03, PNorm = 90.2208, GNorm = 0.3777, lr_0 = 6.4931e-04
Validation auc = 0.706103
Epoch 260
Validation auc = 0.700469
Epoch 261
Validation auc = 0.699531
Epoch 262
Validation auc = 0.699531
Epoch 263
Validation auc = 0.700469
Epoch 264
Loss = 9.2585e-03, PNorm = 90.2368, GNorm = 0.5905, lr_0 = 6.4376e-04
Validation auc = 0.701408
Epoch 265
Validation auc = 0.699531
Epoch 266
Validation auc = 0.701878
Epoch 267
Validation auc = 0.701408
Epoch 268
Validation auc = 0.699061
Epoch 269
Loss = 7.3944e-03, PNorm = 90.2638, GNorm = 1.3150, lr_0 = 6.3826e-04
Validation auc = 0.694366
Epoch 270
Validation auc = 0.691549
Epoch 271
Validation auc = 0.688732
Epoch 272
Validation auc = 0.687324
Epoch 273
Validation auc = 0.687793
Epoch 274
Loss = 8.1753e-04, PNorm = 90.2884, GNorm = 0.0434, lr_0 = 6.3280e-04
Validation auc = 0.686854
Epoch 275
Validation auc = 0.684977
Epoch 276
Validation auc = 0.684507
Epoch 277
Validation auc = 0.683099
Epoch 278
Validation auc = 0.681690
Epoch 279
Loss = 4.2269e-03, PNorm = 90.3048, GNorm = 0.5429, lr_0 = 6.2739e-04
Validation auc = 0.683568
Epoch 280
Validation auc = 0.685915
Epoch 281
Validation auc = 0.686854
Epoch 282
Validation auc = 0.684038
Epoch 283
Validation auc = 0.688263
Epoch 284
Loss = 1.8186e-02, PNorm = 90.3291, GNorm = 2.7324, lr_0 = 6.2203e-04
Validation auc = 0.688263
Epoch 285
Validation auc = 0.684977
Epoch 286
Validation auc = 0.687324
Epoch 287
Validation auc = 0.689202
Epoch 288
Validation auc = 0.691549
Epoch 289
Loss = 1.5974e-03, PNorm = 90.3469, GNorm = 0.0535, lr_0 = 6.1671e-04
Validation auc = 0.692019
Epoch 290
Validation auc = 0.692958
Epoch 291
Validation auc = 0.692958
Epoch 292
Validation auc = 0.691080
Epoch 293
Validation auc = 0.688732
Epoch 294
Loss = 8.5579e-04, PNorm = 90.3705, GNorm = 0.0221, lr_0 = 6.1144e-04
Validation auc = 0.688263
Epoch 295
Validation auc = 0.687793
Epoch 296
Validation auc = 0.689671
Epoch 297
Validation auc = 0.690141
Epoch 298
Validation auc = 0.689202
Epoch 299
Loss = 1.8097e-03, PNorm = 90.3911, GNorm = 0.0924, lr_0 = 6.0621e-04
Validation auc = 0.686854
Model 0 best validation auc = 0.738498 on epoch 256
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.801927
Ensemble test auc = 0.801927
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.654460
Epoch 1
Validation auc = 0.742723
Epoch 2
Validation auc = 0.766197
Epoch 3
Validation auc = 0.748357
Epoch 4
Loss = 6.6649e-01, PNorm = 86.2172, GNorm = 7.4701, lr_0 = 6.5000e-04
Validation auc = 0.753991
Epoch 5
Validation auc = 0.719249
Epoch 6
Validation auc = 0.701408
Epoch 7
Validation auc = 0.703756
Epoch 8
Validation auc = 0.730516
Epoch 9
Loss = 2.2387e-01, PNorm = 86.5486, GNorm = 3.4864, lr_0 = 9.9743e-04
Validation auc = 0.730047
Epoch 10
Validation auc = 0.729108
Epoch 11
Validation auc = 0.733333
Epoch 12
Validation auc = 0.728169
Epoch 13
Validation auc = 0.691080
Epoch 14
Loss = 1.3212e-01, PNorm = 86.8673, GNorm = 2.8492, lr_0 = 9.8890e-04
Validation auc = 0.677934
Epoch 15
Validation auc = 0.708920
Epoch 16
Validation auc = 0.704225
Epoch 17
Validation auc = 0.688263
Epoch 18
Validation auc = 0.664789
Epoch 19
Loss = 3.1602e-01, PNorm = 87.1128, GNorm = 4.7082, lr_0 = 9.8045e-04
Validation auc = 0.707042
Epoch 20
Validation auc = 0.706103
Epoch 21
Validation auc = 0.701408
Epoch 22
Validation auc = 0.703286
Epoch 23
Validation auc = 0.698122
Epoch 24
Loss = 6.4910e-01, PNorm = 87.2994, GNorm = 17.5330, lr_0 = 9.7207e-04
Validation auc = 0.707042
Epoch 25
Validation auc = 0.695775
Epoch 26
Validation auc = 0.671831
Epoch 27
Validation auc = 0.682629
Epoch 28
Validation auc = 0.688732
Epoch 29
Loss = 3.6374e-01, PNorm = 87.4700, GNorm = 3.5793, lr_0 = 9.6376e-04
Validation auc = 0.697183
Epoch 30
Validation auc = 0.698122
Epoch 31
Validation auc = 0.694366
Epoch 32
Validation auc = 0.691080
Epoch 33
Validation auc = 0.703286
Epoch 34
Loss = 2.6107e-01, PNorm = 87.6542, GNorm = 2.6779, lr_0 = 9.5552e-04
Validation auc = 0.702347
Epoch 35
Validation auc = 0.715023
Epoch 36
Validation auc = 0.722066
Epoch 37
Validation auc = 0.704695
Epoch 38
Validation auc = 0.663850
Epoch 39
Loss = 9.8976e-02, PNorm = 87.7743, GNorm = 2.2936, lr_0 = 9.4735e-04
Validation auc = 0.663850
Epoch 40
Validation auc = 0.699061
Epoch 41
Validation auc = 0.719718
Epoch 42
Validation auc = 0.730986
Epoch 43
Validation auc = 0.723944
Epoch 44
Loss = 2.6499e-02, PNorm = 87.8761, GNorm = 1.7782, lr_0 = 9.3925e-04
Validation auc = 0.717840
Epoch 45
Validation auc = 0.709390
Epoch 46
Validation auc = 0.702817
Epoch 47
Validation auc = 0.702347
Epoch 48
Validation auc = 0.699061
Epoch 49
Loss = 8.8377e-02, PNorm = 87.9848, GNorm = 2.0275, lr_0 = 9.3122e-04
Validation auc = 0.676056
Epoch 50
Validation auc = 0.674178
Epoch 51
Validation auc = 0.707042
Epoch 52
Validation auc = 0.723005
Epoch 53
Validation auc = 0.721596
Epoch 54
Loss = 2.7060e-02, PNorm = 88.0762, GNorm = 0.5143, lr_0 = 9.2326e-04
Validation auc = 0.722066
Epoch 55
Validation auc = 0.715962
Epoch 56
Validation auc = 0.701878
Epoch 57
Validation auc = 0.693427
Epoch 58
Validation auc = 0.680751
Epoch 59
Loss = 1.7997e-01, PNorm = 88.1602, GNorm = 5.5923, lr_0 = 9.1537e-04
Validation auc = 0.679343
Epoch 60
Validation auc = 0.680751
Epoch 61
Validation auc = 0.709859
Epoch 62
Validation auc = 0.723474
Epoch 63
Validation auc = 0.730986
Epoch 64
Loss = 5.9457e-02, PNorm = 88.2514, GNorm = 1.6724, lr_0 = 9.0754e-04
Validation auc = 0.716432
Epoch 65
Validation auc = 0.719249
Epoch 66
Validation auc = 0.731455
Epoch 67
Validation auc = 0.710798
Epoch 68
Validation auc = 0.690141
Epoch 69
Loss = 4.7351e-02, PNorm = 88.3537, GNorm = 1.0601, lr_0 = 8.9978e-04
Validation auc = 0.684977
Epoch 70
Validation auc = 0.678404
Epoch 71
Validation auc = 0.681690
Epoch 72
Validation auc = 0.669484
Epoch 73
Validation auc = 0.667136
Epoch 74
Loss = 1.3182e-01, PNorm = 88.4315, GNorm = 2.6817, lr_0 = 8.9209e-04
Validation auc = 0.675117
Epoch 75
Validation auc = 0.684038
Epoch 76
Validation auc = 0.676995
Epoch 77
Validation auc = 0.669953
Epoch 78
Validation auc = 0.679812
Epoch 79
Loss = 8.4560e-02, PNorm = 88.5207, GNorm = 2.6126, lr_0 = 8.8447e-04
Validation auc = 0.686854
Epoch 80
Validation auc = 0.690141
Epoch 81
Validation auc = 0.688263
Epoch 82
Validation auc = 0.685446
Epoch 83
Validation auc = 0.670892
Epoch 84
Loss = 1.2529e-01, PNorm = 88.6242, GNorm = 6.6572, lr_0 = 8.7691e-04
Validation auc = 0.663380
Epoch 85
Validation auc = 0.668075
Epoch 86
Validation auc = 0.680282
Epoch 87
Validation auc = 0.684507
Epoch 88
Validation auc = 0.692958
Epoch 89
Loss = 8.2880e-03, PNorm = 88.7521, GNorm = 0.3646, lr_0 = 8.6941e-04
Validation auc = 0.695775
Epoch 90
Validation auc = 0.694366
Epoch 91
Validation auc = 0.688732
Epoch 92
Validation auc = 0.686385
Epoch 93
Validation auc = 0.684038
Epoch 94
Loss = 3.5735e-02, PNorm = 88.9716, GNorm = 1.6452, lr_0 = 8.6198e-04
Validation auc = 0.692019
Epoch 95
Validation auc = 0.707042
Epoch 96
Validation auc = 0.719249
Epoch 97
Validation auc = 0.725352
Epoch 98
Validation auc = 0.726291
Epoch 99
Loss = 4.0692e-02, PNorm = 89.1268, GNorm = 1.9175, lr_0 = 8.5461e-04
Validation auc = 0.727700
Epoch 100
Validation auc = 0.731925
Epoch 101
Validation auc = 0.711737
Epoch 102
Validation auc = 0.699531
Epoch 103
Validation auc = 0.691549
Epoch 104
Loss = 2.5929e-02, PNorm = 89.2596, GNorm = 3.4440, lr_0 = 8.4730e-04
Validation auc = 0.688732
Epoch 105
Validation auc = 0.682160
Epoch 106
Validation auc = 0.680282
Epoch 107
Validation auc = 0.685446
Epoch 108
Validation auc = 0.695775
Epoch 109
Loss = 6.2364e-02, PNorm = 89.3603, GNorm = 3.7050, lr_0 = 8.4006e-04
Validation auc = 0.692019
Epoch 110
Validation auc = 0.690610
Epoch 111
Validation auc = 0.681690
Epoch 112
Validation auc = 0.676526
Epoch 113
Validation auc = 0.677934
Epoch 114
Loss = 1.2305e-02, PNorm = 89.4332, GNorm = 0.4860, lr_0 = 8.3288e-04
Validation auc = 0.683568
Epoch 115
Validation auc = 0.686854
Epoch 116
Validation auc = 0.688263
Epoch 117
Validation auc = 0.688732
Epoch 118
Validation auc = 0.700469
Epoch 119
Loss = 2.1589e-02, PNorm = 89.5039, GNorm = 0.3070, lr_0 = 8.2576e-04
Validation auc = 0.715023
Epoch 120
Validation auc = 0.724413
Epoch 121
Validation auc = 0.720657
Epoch 122
Validation auc = 0.723474
Epoch 123
Validation auc = 0.729108
Epoch 124
Loss = 3.3755e-02, PNorm = 89.5781, GNorm = 1.1239, lr_0 = 8.1870e-04
Validation auc = 0.738498
Epoch 125
Validation auc = 0.740376
Epoch 126
Validation auc = 0.731455
Epoch 127
Validation auc = 0.718779
Epoch 128
Validation auc = 0.707042
Epoch 129
Loss = 2.9634e-02, PNorm = 89.6474, GNorm = 1.0576, lr_0 = 8.1170e-04
Validation auc = 0.703286
Epoch 130
Validation auc = 0.708920
Epoch 131
Validation auc = 0.707042
Epoch 132
Validation auc = 0.701408
Epoch 133
Validation auc = 0.698592
Epoch 134
Loss = 7.7206e-02, PNorm = 89.7138, GNorm = 2.4891, lr_0 = 8.0476e-04
Validation auc = 0.685446
Epoch 135
Validation auc = 0.676526
Epoch 136
Validation auc = 0.678404
Epoch 137
Validation auc = 0.688263
Epoch 138
Validation auc = 0.694366
Epoch 139
Loss = 5.7906e-03, PNorm = 89.7577, GNorm = 0.7022, lr_0 = 7.9788e-04
Validation auc = 0.700469
Epoch 140
Validation auc = 0.701408
Epoch 141
Validation auc = 0.708451
Epoch 142
Validation auc = 0.705164
Epoch 143
Validation auc = 0.707512
Epoch 144
Loss = 2.3554e-02, PNorm = 89.8099, GNorm = 2.0949, lr_0 = 7.9106e-04
Validation auc = 0.711268
Epoch 145
Validation auc = 0.710798
Epoch 146
Validation auc = 0.707042
Epoch 147
Validation auc = 0.707042
Epoch 148
Validation auc = 0.707042
Epoch 149
Loss = 2.9370e-02, PNorm = 89.8860, GNorm = 0.4593, lr_0 = 7.8430e-04
Validation auc = 0.712207
Epoch 150
Validation auc = 0.715962
Epoch 151
Validation auc = 0.712207
Epoch 152
Validation auc = 0.711737
Epoch 153
Validation auc = 0.713146
Epoch 154
Loss = 9.8305e-03, PNorm = 89.9816, GNorm = 0.2380, lr_0 = 7.7759e-04
Validation auc = 0.713615
Epoch 155
Validation auc = 0.715023
Epoch 156
Validation auc = 0.714554
Epoch 157
Validation auc = 0.717840
Epoch 158
Validation auc = 0.715493
Epoch 159
Loss = 1.9879e-02, PNorm = 90.0629, GNorm = 0.0607, lr_0 = 7.7095e-04
Validation auc = 0.711737
Epoch 160
Validation auc = 0.713146
Epoch 161
Validation auc = 0.703756
Epoch 162
Validation auc = 0.694836
Epoch 163
Validation auc = 0.694366
Epoch 164
Loss = 1.0885e-02, PNorm = 90.1357, GNorm = 0.6688, lr_0 = 7.6436e-04
Validation auc = 0.695775
Epoch 165
Validation auc = 0.689202
Epoch 166
Validation auc = 0.685915
Epoch 167
Validation auc = 0.694366
Epoch 168
Validation auc = 0.698122
Epoch 169
Loss = 3.3592e-02, PNorm = 90.2033, GNorm = 0.3517, lr_0 = 7.5782e-04
Validation auc = 0.697183
Epoch 170
Validation auc = 0.700939
Epoch 171
Validation auc = 0.698122
Epoch 172
Validation auc = 0.693897
Epoch 173
Validation auc = 0.688263
Epoch 174
Loss = 1.8934e-02, PNorm = 90.2617, GNorm = 1.3910, lr_0 = 7.5134e-04
Validation auc = 0.681221
Epoch 175
Validation auc = 0.679812
Epoch 176
Validation auc = 0.680282
Epoch 177
Validation auc = 0.683099
Epoch 178
Validation auc = 0.684977
Epoch 179
Loss = 1.4140e-02, PNorm = 90.3083, GNorm = 1.4112, lr_0 = 7.4492e-04
Validation auc = 0.685915
Epoch 180
Validation auc = 0.679812
Epoch 181
Validation auc = 0.678873
Epoch 182
Validation auc = 0.675117
Epoch 183
Validation auc = 0.672770
Epoch 184
Loss = 3.2786e-02, PNorm = 90.3896, GNorm = 0.2371, lr_0 = 7.3855e-04
Validation auc = 0.668075
Epoch 185
Validation auc = 0.668545
Epoch 186
Validation auc = 0.676995
Epoch 187
Validation auc = 0.681690
Epoch 188
Validation auc = 0.689671
Epoch 189
Loss = 1.5448e-02, PNorm = 90.4486, GNorm = 0.1145, lr_0 = 7.3224e-04
Validation auc = 0.701878
Epoch 190
Validation auc = 0.708920
Epoch 191
Validation auc = 0.716432
Epoch 192
Validation auc = 0.720657
Epoch 193
Validation auc = 0.721596
Epoch 194
Loss = 1.9057e-03, PNorm = 90.4985, GNorm = 0.1129, lr_0 = 7.2598e-04
Validation auc = 0.720188
Epoch 195
Validation auc = 0.714085
Epoch 196
Validation auc = 0.707042
Epoch 197
Validation auc = 0.697183
Epoch 198
Validation auc = 0.698592
Epoch 199
Loss = 4.0225e-02, PNorm = 90.6024, GNorm = 0.4605, lr_0 = 7.1977e-04
Validation auc = 0.706103
Epoch 200
Validation auc = 0.707042
Epoch 201
Validation auc = 0.709859
Epoch 202
Validation auc = 0.706573
Epoch 203
Validation auc = 0.700000
Epoch 204
Loss = 3.3577e-03, PNorm = 90.6649, GNorm = 0.2483, lr_0 = 7.1362e-04
Validation auc = 0.694836
Epoch 205
Validation auc = 0.691549
Epoch 206
Validation auc = 0.690141
Epoch 207
Validation auc = 0.687793
Epoch 208
Validation auc = 0.685446
Epoch 209
Loss = 1.7155e-02, PNorm = 90.7121, GNorm = 1.5680, lr_0 = 7.0752e-04
Validation auc = 0.687793
Epoch 210
Validation auc = 0.692019
Epoch 211
Validation auc = 0.692019
Epoch 212
Validation auc = 0.694836
Epoch 213
Validation auc = 0.697183
Epoch 214
Loss = 2.8557e-02, PNorm = 90.7642, GNorm = 1.7711, lr_0 = 7.0147e-04
Validation auc = 0.697183
Epoch 215
Validation auc = 0.695775
Epoch 216
Validation auc = 0.694836
Epoch 217
Validation auc = 0.691080
Epoch 218
Validation auc = 0.691549
Epoch 219
Loss = 3.5259e-03, PNorm = 90.8052, GNorm = 0.1737, lr_0 = 6.9548e-04
Validation auc = 0.693897
Epoch 220
Validation auc = 0.696244
Epoch 221
Validation auc = 0.700000
Epoch 222
Validation auc = 0.701878
Epoch 223
Validation auc = 0.706573
Epoch 224
Loss = 1.1423e-03, PNorm = 90.8327, GNorm = 0.0349, lr_0 = 6.8953e-04
Validation auc = 0.710329
Epoch 225
Validation auc = 0.710329
Epoch 226
Validation auc = 0.713146
Epoch 227
Validation auc = 0.713615
Epoch 228
Validation auc = 0.712676
Epoch 229
Loss = 9.9863e-04, PNorm = 90.8461, GNorm = 0.0867, lr_0 = 6.8364e-04
Validation auc = 0.712676
Epoch 230
Validation auc = 0.710329
Epoch 231
Validation auc = 0.710798
Epoch 232
Validation auc = 0.707981
Epoch 233
Validation auc = 0.704225
Epoch 234
Loss = 2.4264e-03, PNorm = 90.8642, GNorm = 0.0452, lr_0 = 6.7779e-04
Validation auc = 0.702347
Epoch 235
Validation auc = 0.704695
Epoch 236
Validation auc = 0.705164
Epoch 237
Validation auc = 0.704695
Epoch 238
Validation auc = 0.705164
Epoch 239
Loss = 2.0663e-03, PNorm = 90.8820, GNorm = 0.2120, lr_0 = 6.7200e-04
Validation auc = 0.704695
Epoch 240
Validation auc = 0.707512
Epoch 241
Validation auc = 0.708920
Epoch 242
Validation auc = 0.707981
Epoch 243
Validation auc = 0.713146
Epoch 244
Loss = 4.9321e-04, PNorm = 90.9058, GNorm = 0.0178, lr_0 = 6.6625e-04
Validation auc = 0.712676
Epoch 245
Validation auc = 0.712676
Epoch 246
Validation auc = 0.713146
Epoch 247
Validation auc = 0.712676
Epoch 248
Validation auc = 0.715493
Epoch 249
Loss = 1.3975e-03, PNorm = 90.9275, GNorm = 0.0951, lr_0 = 6.6056e-04
Validation auc = 0.715493
Epoch 250
Validation auc = 0.716901
Epoch 251
Validation auc = 0.718310
Epoch 252
Validation auc = 0.718310
Epoch 253
Validation auc = 0.717840
Epoch 254
Loss = 6.5991e-02, PNorm = 90.9578, GNorm = 5.2211, lr_0 = 6.5491e-04
Validation auc = 0.720188
Epoch 255
Validation auc = 0.721127
Epoch 256
Validation auc = 0.723005
Epoch 257
Validation auc = 0.718779
Epoch 258
Validation auc = 0.711737
Epoch 259
Loss = 2.7664e-03, PNorm = 91.0427, GNorm = 0.3369, lr_0 = 6.4931e-04
Validation auc = 0.714085
Epoch 260
Validation auc = 0.714085
Epoch 261
Validation auc = 0.712676
Epoch 262
Validation auc = 0.715493
Epoch 263
Validation auc = 0.713146
Epoch 264
Loss = 1.9617e-03, PNorm = 91.1299, GNorm = 0.1677, lr_0 = 6.4376e-04
Validation auc = 0.719249
Epoch 265
Validation auc = 0.724883
Epoch 266
Validation auc = 0.729108
Epoch 267
Validation auc = 0.730047
Epoch 268
Validation auc = 0.731455
Epoch 269
Loss = 4.0201e-04, PNorm = 91.1778, GNorm = 0.0173, lr_0 = 6.3826e-04
Validation auc = 0.733333
Epoch 270
Validation auc = 0.736150
Epoch 271
Validation auc = 0.735681
Epoch 272
Validation auc = 0.735681
Epoch 273
Validation auc = 0.735211
Epoch 274
Loss = 2.7913e-03, PNorm = 91.2045, GNorm = 0.3360, lr_0 = 6.3280e-04
Validation auc = 0.735681
Epoch 275
Validation auc = 0.735681
Epoch 276
Validation auc = 0.735211
Epoch 277
Validation auc = 0.735211
Epoch 278
Validation auc = 0.732864
Epoch 279
Loss = 4.2155e-03, PNorm = 91.2296, GNorm = 0.2322, lr_0 = 6.2739e-04
Validation auc = 0.730986
Epoch 280
Validation auc = 0.730986
Epoch 281
Validation auc = 0.732394
Epoch 282
Validation auc = 0.730986
Epoch 283
Validation auc = 0.729577
Epoch 284
Loss = 5.5647e-04, PNorm = 91.2485, GNorm = 0.0304, lr_0 = 6.2203e-04
Validation auc = 0.729108
Epoch 285
Validation auc = 0.729577
Epoch 286
Validation auc = 0.729108
Epoch 287
Validation auc = 0.726291
Epoch 288
Validation auc = 0.728638
Epoch 289
Loss = 1.2578e-03, PNorm = 91.2700, GNorm = 0.0580, lr_0 = 6.1671e-04
Validation auc = 0.724413
Epoch 290
Validation auc = 0.723944
Epoch 291
Validation auc = 0.722535
Epoch 292
Validation auc = 0.721596
Epoch 293
Validation auc = 0.720188
Epoch 294
Loss = 9.5805e-04, PNorm = 91.2980, GNorm = 0.0050, lr_0 = 6.1144e-04
Validation auc = 0.720657
Epoch 295
Validation auc = 0.719249
Epoch 296
Validation auc = 0.717371
Epoch 297
Validation auc = 0.715962
Epoch 298
Validation auc = 0.716432
Epoch 299
Loss = 1.1516e-03, PNorm = 91.3289, GNorm = 0.0370, lr_0 = 6.0621e-04
Validation auc = 0.716901
Model 0 best validation auc = 0.766197 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.883805
Ensemble test auc = 0.883805
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.668075
Epoch 1
Validation auc = 0.711737
Epoch 2
Validation auc = 0.722066
Epoch 3
Validation auc = 0.722066
Epoch 4
Loss = 6.4380e-01, PNorm = 86.2045, GNorm = 6.2563, lr_0 = 6.5000e-04
Validation auc = 0.729577
Epoch 5
Validation auc = 0.728169
Epoch 6
Validation auc = 0.695305
Epoch 7
Validation auc = 0.726291
Epoch 8
Validation auc = 0.736150
Epoch 9
Loss = 2.3791e-01, PNorm = 86.5118, GNorm = 3.3199, lr_0 = 9.9743e-04
Validation auc = 0.736620
Epoch 10
Validation auc = 0.686385
Epoch 11
Validation auc = 0.618779
Epoch 12
Validation auc = 0.597183
Epoch 13
Validation auc = 0.682160
Epoch 14
Loss = 4.1566e-01, PNorm = 86.8031, GNorm = 8.6588, lr_0 = 9.8890e-04
Validation auc = 0.709859
Epoch 15
Validation auc = 0.721596
Epoch 16
Validation auc = 0.725822
Epoch 17
Validation auc = 0.726761
Epoch 18
Validation auc = 0.720657
Epoch 19
Loss = 3.2215e-01, PNorm = 87.0069, GNorm = 3.7253, lr_0 = 9.8045e-04
Validation auc = 0.706573
Epoch 20
Validation auc = 0.701878
Epoch 21
Validation auc = 0.695775
Epoch 22
Validation auc = 0.675587
Epoch 23
Validation auc = 0.703286
Epoch 24
Loss = 1.1815e-01, PNorm = 87.1483, GNorm = 3.2549, lr_0 = 9.7207e-04
Validation auc = 0.715023
Epoch 25
Validation auc = 0.694366
Epoch 26
Validation auc = 0.694366
Epoch 27
Validation auc = 0.676995
Epoch 28
Validation auc = 0.679812
Epoch 29
Loss = 8.9339e-02, PNorm = 87.2496, GNorm = 2.0975, lr_0 = 9.6376e-04
Validation auc = 0.687324
Epoch 30
Validation auc = 0.693427
Epoch 31
Validation auc = 0.718310
Epoch 32
Validation auc = 0.714554
Epoch 33
Validation auc = 0.692958
Epoch 34
Loss = 8.4326e-02, PNorm = 87.3674, GNorm = 1.1051, lr_0 = 9.5552e-04
Validation auc = 0.669953
Epoch 35
Validation auc = 0.677465
Epoch 36
Validation auc = 0.685446
Epoch 37
Validation auc = 0.716432
Epoch 38
Validation auc = 0.742723
Epoch 39
Loss = 2.1973e-01, PNorm = 87.4440, GNorm = 4.3631, lr_0 = 9.4735e-04
Validation auc = 0.737089
Epoch 40
Validation auc = 0.718310
Epoch 41
Validation auc = 0.701878
Epoch 42
Validation auc = 0.692019
Epoch 43
Validation auc = 0.700939
Epoch 44
Loss = 1.3764e-01, PNorm = 87.5477, GNorm = 3.2278, lr_0 = 9.3925e-04
Validation auc = 0.717371
Epoch 45
Validation auc = 0.729108
Epoch 46
Validation auc = 0.730047
Epoch 47
Validation auc = 0.731925
Epoch 48
Validation auc = 0.733803
Epoch 49
Loss = 4.1675e-02, PNorm = 87.6615, GNorm = 1.8989, lr_0 = 9.3122e-04
Validation auc = 0.733333
Epoch 50
Validation auc = 0.726761
Epoch 51
Validation auc = 0.707512
Epoch 52
Validation auc = 0.699061
Epoch 53
Validation auc = 0.690141
Epoch 54
Loss = 2.2917e-01, PNorm = 87.7659, GNorm = 7.9491, lr_0 = 9.2326e-04
Validation auc = 0.669014
Epoch 55
Validation auc = 0.659155
Epoch 56
Validation auc = 0.666667
Epoch 57
Validation auc = 0.673239
Epoch 58
Validation auc = 0.666667
Epoch 59
Loss = 1.4061e-01, PNorm = 87.8968, GNorm = 2.8650, lr_0 = 9.1537e-04
Validation auc = 0.664789
Epoch 60
Validation auc = 0.663850
Epoch 61
Validation auc = 0.662911
Epoch 62
Validation auc = 0.656808
Epoch 63
Validation auc = 0.646009
Epoch 64
Loss = 1.6113e-01, PNorm = 88.0159, GNorm = 7.6928, lr_0 = 9.0754e-04
Validation auc = 0.652113
Epoch 65
Validation auc = 0.680751
Epoch 66
Validation auc = 0.698122
Epoch 67
Validation auc = 0.711737
Epoch 68
Validation auc = 0.700469
Epoch 69
Loss = 3.6478e-02, PNorm = 88.1131, GNorm = 2.3224, lr_0 = 8.9978e-04
Validation auc = 0.687793
Epoch 70
Validation auc = 0.686385
Epoch 71
Validation auc = 0.684977
Epoch 72
Validation auc = 0.680282
Epoch 73
Validation auc = 0.673239
Epoch 74
Loss = 3.5036e-02, PNorm = 88.2097, GNorm = 1.0764, lr_0 = 8.9209e-04
Validation auc = 0.667136
Epoch 75
Validation auc = 0.671362
Epoch 76
Validation auc = 0.673239
Epoch 77
Validation auc = 0.671831
Epoch 78
Validation auc = 0.667136
Epoch 79
Loss = 2.1862e-02, PNorm = 88.2624, GNorm = 1.3032, lr_0 = 8.8447e-04
Validation auc = 0.673239
Epoch 80
Validation auc = 0.677934
Epoch 81
Validation auc = 0.678873
Epoch 82
Validation auc = 0.680751
Epoch 83
Validation auc = 0.676526
Epoch 84
Loss = 2.1214e-02, PNorm = 88.3351, GNorm = 1.3827, lr_0 = 8.7691e-04
Validation auc = 0.666667
Epoch 85
Validation auc = 0.669014
Epoch 86
Validation auc = 0.671362
Epoch 87
Validation auc = 0.671831
Epoch 88
Validation auc = 0.670423
Epoch 89
Loss = 2.4211e-02, PNorm = 88.4255, GNorm = 1.2204, lr_0 = 8.6941e-04
Validation auc = 0.669484
Epoch 90
Validation auc = 0.670892
Epoch 91
Validation auc = 0.679343
Epoch 92
Validation auc = 0.685915
Epoch 93
Validation auc = 0.692488
Epoch 94
Loss = 9.2355e-02, PNorm = 88.4852, GNorm = 3.6548, lr_0 = 8.6198e-04
Validation auc = 0.696244
Epoch 95
Validation auc = 0.693427
Epoch 96
Validation auc = 0.683568
Epoch 97
Validation auc = 0.684977
Epoch 98
Validation auc = 0.679343
Epoch 99
Loss = 1.6390e-02, PNorm = 88.5345, GNorm = 0.4124, lr_0 = 8.5461e-04
Validation auc = 0.677934
Epoch 100
Validation auc = 0.676526
Epoch 101
Validation auc = 0.671831
Epoch 102
Validation auc = 0.669014
Epoch 103
Validation auc = 0.675117
Epoch 104
Loss = 1.0301e-01, PNorm = 88.5899, GNorm = 5.7126, lr_0 = 8.4730e-04
Validation auc = 0.671362
Epoch 105
Validation auc = 0.669953
Epoch 106
Validation auc = 0.671831
Epoch 107
Validation auc = 0.677465
Epoch 108
Validation auc = 0.681221
Epoch 109
Loss = 4.5121e-02, PNorm = 88.6436, GNorm = 3.1643, lr_0 = 8.4006e-04
Validation auc = 0.680282
Epoch 110
Validation auc = 0.689202
Epoch 111
Validation auc = 0.685915
Epoch 112
Validation auc = 0.685446
Epoch 113
Validation auc = 0.691080
Epoch 114
Loss = 7.7328e-03, PNorm = 88.7052, GNorm = 0.0782, lr_0 = 8.3288e-04
Validation auc = 0.694366
Epoch 115
Validation auc = 0.700469
Epoch 116
Validation auc = 0.698592
Epoch 117
Validation auc = 0.695305
Epoch 118
Validation auc = 0.690610
Epoch 119
Loss = 8.4071e-03, PNorm = 88.7854, GNorm = 0.3270, lr_0 = 8.2576e-04
Validation auc = 0.682629
Epoch 120
Validation auc = 0.680751
Epoch 121
Validation auc = 0.680751
Epoch 122
Validation auc = 0.672770
Epoch 123
Validation auc = 0.679812
Epoch 124
Loss = 1.7006e-02, PNorm = 88.8598, GNorm = 1.2801, lr_0 = 8.1870e-04
Validation auc = 0.685446
Epoch 125
Validation auc = 0.690610
Epoch 126
Validation auc = 0.691080
Epoch 127
Validation auc = 0.685915
Epoch 128
Validation auc = 0.678873
Epoch 129
Loss = 2.5226e-02, PNorm = 88.9360, GNorm = 0.1631, lr_0 = 8.1170e-04
Validation auc = 0.682160
Epoch 130
Validation auc = 0.669953
Epoch 131
Validation auc = 0.655399
Epoch 132
Validation auc = 0.667136
Epoch 133
Validation auc = 0.671831
Epoch 134
Loss = 1.9287e-02, PNorm = 89.0255, GNorm = 0.2517, lr_0 = 8.0476e-04
Validation auc = 0.681221
Epoch 135
Validation auc = 0.689671
Epoch 136
Validation auc = 0.696244
Epoch 137
Validation auc = 0.700000
Epoch 138
Validation auc = 0.698122
Epoch 139
Loss = 3.3784e-02, PNorm = 89.1117, GNorm = 3.3711, lr_0 = 7.9788e-04
Validation auc = 0.701878
Epoch 140
Validation auc = 0.698592
Epoch 141
Validation auc = 0.693427
Epoch 142
Validation auc = 0.689671
Epoch 143
Validation auc = 0.692488
Epoch 144
Loss = 3.1630e-02, PNorm = 89.2001, GNorm = 3.5915, lr_0 = 7.9106e-04
Validation auc = 0.694366
Epoch 145
Validation auc = 0.690610
Epoch 146
Validation auc = 0.690610
Epoch 147
Validation auc = 0.690610
Epoch 148
Validation auc = 0.690610
Epoch 149
Loss = 4.3771e-03, PNorm = 89.2694, GNorm = 0.1494, lr_0 = 7.8430e-04
Validation auc = 0.692488
Epoch 150
Validation auc = 0.693427
Epoch 151
Validation auc = 0.699061
Epoch 152
Validation auc = 0.701878
Epoch 153
Validation auc = 0.706103
Epoch 154
Loss = 1.7395e-02, PNorm = 89.3543, GNorm = 0.1935, lr_0 = 7.7759e-04
Validation auc = 0.706103
Epoch 155
Validation auc = 0.714085
Epoch 156
Validation auc = 0.707042
Epoch 157
Validation auc = 0.709390
Epoch 158
Validation auc = 0.702347
Epoch 159
Loss = 9.5459e-03, PNorm = 89.4480, GNorm = 1.5082, lr_0 = 7.7095e-04
Validation auc = 0.697653
Epoch 160
Validation auc = 0.696244
Epoch 161
Validation auc = 0.689202
Epoch 162
Validation auc = 0.685915
Epoch 163
Validation auc = 0.690141
Epoch 164
Loss = 3.4864e-02, PNorm = 89.5423, GNorm = 3.5169, lr_0 = 7.6436e-04
Validation auc = 0.687324
Epoch 165
Validation auc = 0.677934
Epoch 166
Validation auc = 0.674178
Epoch 167
Validation auc = 0.675117
Epoch 168
Validation auc = 0.679812
Epoch 169
Loss = 1.1088e-02, PNorm = 89.6407, GNorm = 0.6920, lr_0 = 7.5782e-04
Validation auc = 0.687324
Epoch 170
Validation auc = 0.692488
Epoch 171
Validation auc = 0.698122
Epoch 172
Validation auc = 0.692488
Epoch 173
Validation auc = 0.689671
Epoch 174
Loss = 1.0235e-02, PNorm = 89.7513, GNorm = 0.3948, lr_0 = 7.5134e-04
Validation auc = 0.687793
Epoch 175
Validation auc = 0.691080
Epoch 176
Validation auc = 0.696244
Epoch 177
Validation auc = 0.702817
Epoch 178
Validation auc = 0.701408
Epoch 179
Loss = 1.7653e-02, PNorm = 89.8230, GNorm = 1.2020, lr_0 = 7.4492e-04
Validation auc = 0.695305
Epoch 180
Validation auc = 0.697183
Epoch 181
Validation auc = 0.689671
Epoch 182
Validation auc = 0.683568
Epoch 183
Validation auc = 0.685915
Epoch 184
Loss = 1.8522e-02, PNorm = 89.9195, GNorm = 0.0576, lr_0 = 7.3855e-04
Validation auc = 0.699061
Epoch 185
Validation auc = 0.706103
Epoch 186
Validation auc = 0.707042
Epoch 187
Validation auc = 0.708451
Epoch 188
Validation auc = 0.710329
Epoch 189
Loss = 1.3599e-02, PNorm = 89.9855, GNorm = 0.9204, lr_0 = 7.3224e-04
Validation auc = 0.708451
Epoch 190
Validation auc = 0.704695
Epoch 191
Validation auc = 0.701408
Epoch 192
Validation auc = 0.698592
Epoch 193
Validation auc = 0.696244
Epoch 194
Loss = 3.6427e-03, PNorm = 90.0322, GNorm = 0.1415, lr_0 = 7.2598e-04
Validation auc = 0.695305
Epoch 195
Validation auc = 0.695305
Epoch 196
Validation auc = 0.694836
Epoch 197
Validation auc = 0.695775
Epoch 198
Validation auc = 0.696244
Epoch 199
Loss = 2.5111e-03, PNorm = 90.0624, GNorm = 0.0597, lr_0 = 7.1977e-04
Validation auc = 0.696714
Epoch 200
Validation auc = 0.698122
Epoch 201
Validation auc = 0.705634
Epoch 202
Validation auc = 0.706103
Epoch 203
Validation auc = 0.707981
Epoch 204
Loss = 9.0646e-03, PNorm = 90.1013, GNorm = 0.9251, lr_0 = 7.1362e-04
Validation auc = 0.709390
Epoch 205
Validation auc = 0.711268
Epoch 206
Validation auc = 0.712676
Epoch 207
Validation auc = 0.709390
Epoch 208
Validation auc = 0.709859
Epoch 209
Loss = 1.7237e-02, PNorm = 90.1445, GNorm = 1.5988, lr_0 = 7.0752e-04
Validation auc = 0.706573
Epoch 210
Validation auc = 0.707042
Epoch 211
Validation auc = 0.707981
Epoch 212
Validation auc = 0.708920
Epoch 213
Validation auc = 0.712676
Epoch 214
Loss = 1.4894e-02, PNorm = 90.1731, GNorm = 0.3439, lr_0 = 7.0147e-04
Validation auc = 0.708920
Epoch 215
Validation auc = 0.704695
Epoch 216
Validation auc = 0.699061
Epoch 217
Validation auc = 0.701408
Epoch 218
Validation auc = 0.705164
Epoch 219
Loss = 1.9955e-03, PNorm = 90.1974, GNorm = 0.0197, lr_0 = 6.9548e-04
Validation auc = 0.712676
Epoch 220
Validation auc = 0.720188
Epoch 221
Validation auc = 0.726761
Epoch 222
Validation auc = 0.727230
Epoch 223
Validation auc = 0.730516
Epoch 224
Loss = 7.3514e-03, PNorm = 90.2911, GNorm = 0.3653, lr_0 = 6.8953e-04
Validation auc = 0.728169
Epoch 225
Validation auc = 0.715493
Epoch 226
Validation auc = 0.713146
Epoch 227
Validation auc = 0.717840
Epoch 228
Validation auc = 0.711737
Epoch 229
Loss = 7.2361e-03, PNorm = 90.3808, GNorm = 0.7342, lr_0 = 6.8364e-04
Validation auc = 0.709859
Epoch 230
Validation auc = 0.706573
Epoch 231
Validation auc = 0.706573
Epoch 232
Validation auc = 0.705164
Epoch 233
Validation auc = 0.702347
Epoch 234
Loss = 3.3008e-03, PNorm = 90.5066, GNorm = 0.2641, lr_0 = 6.7779e-04
Validation auc = 0.700469
Epoch 235
Validation auc = 0.704225
Epoch 236
Validation auc = 0.702817
Epoch 237
Validation auc = 0.703286
Epoch 238
Validation auc = 0.698592
Epoch 239
Loss = 7.9629e-03, PNorm = 90.5896, GNorm = 0.1845, lr_0 = 6.7200e-04
Validation auc = 0.700469
Epoch 240
Validation auc = 0.700000
Epoch 241
Validation auc = 0.698592
Epoch 242
Validation auc = 0.698592
Epoch 243
Validation auc = 0.697653
Epoch 244
Loss = 9.4651e-02, PNorm = 90.6234, GNorm = 3.6184, lr_0 = 6.6625e-04
Validation auc = 0.699531
Epoch 245
Validation auc = 0.700939
Epoch 246
Validation auc = 0.697183
Epoch 247
Validation auc = 0.701408
Epoch 248
Validation auc = 0.707042
Epoch 249
Loss = 3.5307e-03, PNorm = 90.7069, GNorm = 0.4452, lr_0 = 6.6056e-04
Validation auc = 0.710329
Epoch 250
Validation auc = 0.710329
Epoch 251
Validation auc = 0.710329
Epoch 252
Validation auc = 0.705164
Epoch 253
Validation auc = 0.702817
Epoch 254
Loss = 1.1707e-02, PNorm = 90.7766, GNorm = 0.8074, lr_0 = 6.5491e-04
Validation auc = 0.701408
Epoch 255
Validation auc = 0.700000
Epoch 256
Validation auc = 0.699531
Epoch 257
Validation auc = 0.698122
Epoch 258
Validation auc = 0.697653
Epoch 259
Loss = 4.8646e-03, PNorm = 90.8588, GNorm = 0.1155, lr_0 = 6.4931e-04
Validation auc = 0.692019
Epoch 260
Validation auc = 0.691549
Epoch 261
Validation auc = 0.691080
Epoch 262
Validation auc = 0.692019
Epoch 263
Validation auc = 0.690610
Epoch 264
Loss = 2.6465e-02, PNorm = 90.9323, GNorm = 0.0292, lr_0 = 6.4376e-04
Validation auc = 0.697653
Epoch 265
Validation auc = 0.704695
Epoch 266
Validation auc = 0.707042
Epoch 267
Validation auc = 0.707512
Epoch 268
Validation auc = 0.705164
Epoch 269
Loss = 6.3609e-04, PNorm = 91.0084, GNorm = 0.0179, lr_0 = 6.3826e-04
Validation auc = 0.706573
Epoch 270
Validation auc = 0.705634
Epoch 271
Validation auc = 0.707512
Epoch 272
Validation auc = 0.704225
Epoch 273
Validation auc = 0.699061
Epoch 274
Loss = 2.4420e-03, PNorm = 91.0510, GNorm = 0.0660, lr_0 = 6.3280e-04
Validation auc = 0.697183
Epoch 275
Validation auc = 0.693897
Epoch 276
Validation auc = 0.693427
Epoch 277
Validation auc = 0.696714
Epoch 278
Validation auc = 0.695775
Epoch 279
Loss = 1.5197e-03, PNorm = 91.0825, GNorm = 0.0992, lr_0 = 6.2739e-04
Validation auc = 0.695775
Epoch 280
Validation auc = 0.700469
Epoch 281
Validation auc = 0.701408
Epoch 282
Validation auc = 0.707512
Epoch 283
Validation auc = 0.710798
Epoch 284
Loss = 4.6391e-03, PNorm = 91.1188, GNorm = 1.1055, lr_0 = 6.2203e-04
Validation auc = 0.711737
Epoch 285
Validation auc = 0.715023
Epoch 286
Validation auc = 0.715962
Epoch 287
Validation auc = 0.715962
Epoch 288
Validation auc = 0.714085
Epoch 289
Loss = 3.0811e-04, PNorm = 91.1506, GNorm = 0.0083, lr_0 = 6.1671e-04
Validation auc = 0.712676
Epoch 290
Validation auc = 0.711737
Epoch 291
Validation auc = 0.712207
Epoch 292
Validation auc = 0.710798
Epoch 293
Validation auc = 0.711737
Epoch 294
Loss = 5.3195e-03, PNorm = 91.1710, GNorm = 0.0221, lr_0 = 6.1144e-04
Validation auc = 0.711268
Epoch 295
Validation auc = 0.711268
Epoch 296
Validation auc = 0.713615
Epoch 297
Validation auc = 0.716432
Epoch 298
Validation auc = 0.711268
Epoch 299
Loss = 4.5368e-03, PNorm = 91.2001, GNorm = 0.3245, lr_0 = 6.0621e-04
Validation auc = 0.711737
Model 0 best validation auc = 0.742723 on epoch 38
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.915111
Ensemble test auc = 0.915111
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.692019
Epoch 1
Validation auc = 0.774648
Epoch 2
Validation auc = 0.830986
Epoch 3
Validation auc = 0.792019
Epoch 4
Loss = 6.5573e-01, PNorm = 86.2034, GNorm = 10.9936, lr_0 = 6.5000e-04
Validation auc = 0.784977
Epoch 5
Validation auc = 0.760094
Epoch 6
Validation auc = 0.729577
Epoch 7
Validation auc = 0.765258
Epoch 8
Validation auc = 0.753052
Epoch 9
Loss = 2.4276e-01, PNorm = 86.4978, GNorm = 4.3169, lr_0 = 9.9743e-04
Validation auc = 0.756808
Epoch 10
Validation auc = 0.740376
Epoch 11
Validation auc = 0.730986
Epoch 12
Validation auc = 0.722535
Epoch 13
Validation auc = 0.676056
Epoch 14
Loss = 4.2065e-01, PNorm = 86.7789, GNorm = 3.4759, lr_0 = 9.8890e-04
Validation auc = 0.738028
Epoch 15
Validation auc = 0.717371
Epoch 16
Validation auc = 0.725822
Epoch 17
Validation auc = 0.737559
Epoch 18
Validation auc = 0.739906
Epoch 19
Loss = 1.5025e-01, PNorm = 86.9796, GNorm = 4.3228, lr_0 = 9.8045e-04
Validation auc = 0.725352
Epoch 20
Validation auc = 0.722066
Epoch 21
Validation auc = 0.730516
Epoch 22
Validation auc = 0.755399
Epoch 23
Validation auc = 0.737559
Epoch 24
Loss = 1.8544e-01, PNorm = 87.1123, GNorm = 6.5197, lr_0 = 9.7207e-04
Validation auc = 0.731455
Epoch 25
Validation auc = 0.753521
Epoch 26
Validation auc = 0.754460
Epoch 27
Validation auc = 0.746948
Epoch 28
Validation auc = 0.736150
Epoch 29
Loss = 2.2848e-01, PNorm = 87.2194, GNorm = 3.4325, lr_0 = 9.6376e-04
Validation auc = 0.723944
Epoch 30
Validation auc = 0.722535
Epoch 31
Validation auc = 0.725822
Epoch 32
Validation auc = 0.713615
Epoch 33
Validation auc = 0.707042
Epoch 34
Loss = 9.8259e-02, PNorm = 87.3400, GNorm = 0.4860, lr_0 = 9.5552e-04
Validation auc = 0.732394
Epoch 35
Validation auc = 0.738028
Epoch 36
Validation auc = 0.746009
Epoch 37
Validation auc = 0.757277
Epoch 38
Validation auc = 0.747887
Epoch 39
Loss = 1.2342e-01, PNorm = 87.4671, GNorm = 3.1732, lr_0 = 9.4735e-04
Validation auc = 0.724413
Epoch 40
Validation auc = 0.712207
Epoch 41
Validation auc = 0.718310
Epoch 42
Validation auc = 0.723474
Epoch 43
Validation auc = 0.711268
Epoch 44
Loss = 2.2492e-01, PNorm = 87.5561, GNorm = 6.6578, lr_0 = 9.3925e-04
Validation auc = 0.718310
Epoch 45
Validation auc = 0.730516
Epoch 46
Validation auc = 0.729108
Epoch 47
Validation auc = 0.721596
Epoch 48
Validation auc = 0.712676
Epoch 49
Loss = 7.7187e-02, PNorm = 87.6719, GNorm = 4.0083, lr_0 = 9.3122e-04
Validation auc = 0.717840
Epoch 50
Validation auc = 0.730516
Epoch 51
Validation auc = 0.728638
Epoch 52
Validation auc = 0.723005
Epoch 53
Validation auc = 0.718310
Epoch 54
Loss = 6.1447e-02, PNorm = 87.7811, GNorm = 1.1057, lr_0 = 9.2326e-04
Validation auc = 0.716901
Epoch 55
Validation auc = 0.720188
Epoch 56
Validation auc = 0.718310
Epoch 57
Validation auc = 0.715493
Epoch 58
Validation auc = 0.733803
Epoch 59
Loss = 3.0655e-02, PNorm = 87.8774, GNorm = 0.6233, lr_0 = 9.1537e-04
Validation auc = 0.755399
Epoch 60
Validation auc = 0.767136
Epoch 61
Validation auc = 0.762911
Epoch 62
Validation auc = 0.756338
Epoch 63
Validation auc = 0.766197
Epoch 64
Loss = 1.0439e-01, PNorm = 87.9487, GNorm = 2.7868, lr_0 = 9.0754e-04
Validation auc = 0.757746
Epoch 65
Validation auc = 0.731925
Epoch 66
Validation auc = 0.722535
Epoch 67
Validation auc = 0.710798
Epoch 68
Validation auc = 0.714085
Epoch 69
Loss = 1.4795e-01, PNorm = 88.0183, GNorm = 4.0089, lr_0 = 8.9978e-04
Validation auc = 0.724413
Epoch 70
Validation auc = 0.722066
Epoch 71
Validation auc = 0.713146
Epoch 72
Validation auc = 0.695305
Epoch 73
Validation auc = 0.684507
Epoch 74
Loss = 3.6099e-02, PNorm = 88.0863, GNorm = 1.5662, lr_0 = 8.9209e-04
Validation auc = 0.689202
Epoch 75
Validation auc = 0.700000
Epoch 76
Validation auc = 0.703756
Epoch 77
Validation auc = 0.704695
Epoch 78
Validation auc = 0.709859
Epoch 79
Loss = 2.1565e-02, PNorm = 88.1284, GNorm = 0.2578, lr_0 = 8.8447e-04
Validation auc = 0.725822
Epoch 80
Validation auc = 0.732864
Epoch 81
Validation auc = 0.730986
Epoch 82
Validation auc = 0.732864
Epoch 83
Validation auc = 0.733333
Epoch 84
Loss = 2.2013e-02, PNorm = 88.2146, GNorm = 0.8749, lr_0 = 8.7691e-04
Validation auc = 0.722066
Epoch 85
Validation auc = 0.718779
Epoch 86
Validation auc = 0.714085
Epoch 87
Validation auc = 0.708920
Epoch 88
Validation auc = 0.716432
Epoch 89
Loss = 6.2143e-02, PNorm = 88.2948, GNorm = 0.6112, lr_0 = 8.6941e-04
Validation auc = 0.723005
Epoch 90
Validation auc = 0.727700
Epoch 91
Validation auc = 0.730516
Epoch 92
Validation auc = 0.730516
Epoch 93
Validation auc = 0.725822
Epoch 94
Loss = 1.5604e-02, PNorm = 88.3623, GNorm = 1.2474, lr_0 = 8.6198e-04
Validation auc = 0.718779
Epoch 95
Validation auc = 0.713615
Epoch 96
Validation auc = 0.702817
Epoch 97
Validation auc = 0.697653
Epoch 98
Validation auc = 0.698122
Epoch 99
Loss = 6.5140e-03, PNorm = 88.4202, GNorm = 0.2138, lr_0 = 8.5461e-04
Validation auc = 0.700000
Epoch 100
Validation auc = 0.700000
Epoch 101
Validation auc = 0.699531
Epoch 102
Validation auc = 0.699061
Epoch 103
Validation auc = 0.697653
Epoch 104
Loss = 6.1088e-03, PNorm = 88.4613, GNorm = 0.3280, lr_0 = 8.4730e-04
Validation auc = 0.698592
Epoch 105
Validation auc = 0.700469
Epoch 106
Validation auc = 0.702347
Epoch 107
Validation auc = 0.710798
Epoch 108
Validation auc = 0.710329
Epoch 109
Loss = 5.5655e-03, PNorm = 88.4987, GNorm = 0.3534, lr_0 = 8.4006e-04
Validation auc = 0.712207
Epoch 110
Validation auc = 0.713146
Epoch 111
Validation auc = 0.703756
Epoch 112
Validation auc = 0.699061
Epoch 113
Validation auc = 0.697653
Epoch 114
Loss = 5.9340e-03, PNorm = 88.5326, GNorm = 0.2900, lr_0 = 8.3288e-04
Validation auc = 0.699531
Epoch 115
Validation auc = 0.700000
Epoch 116
Validation auc = 0.702817
Epoch 117
Validation auc = 0.700939
Epoch 118
Validation auc = 0.701408
Epoch 119
Loss = 2.7863e-02, PNorm = 88.5787, GNorm = 0.5776, lr_0 = 8.2576e-04
Validation auc = 0.702347
Epoch 120
Validation auc = 0.703286
Epoch 121
Validation auc = 0.699061
Epoch 122
Validation auc = 0.707981
Epoch 123
Validation auc = 0.707512
Epoch 124
Loss = 5.5007e-03, PNorm = 88.6325, GNorm = 0.2170, lr_0 = 8.1870e-04
Validation auc = 0.705164
Epoch 125
Validation auc = 0.700000
Epoch 126
Validation auc = 0.697183
Epoch 127
Validation auc = 0.693427
Epoch 128
Validation auc = 0.692019
Epoch 129
Loss = 5.8289e-02, PNorm = 88.6862, GNorm = 1.9411, lr_0 = 8.1170e-04
Validation auc = 0.689202
Epoch 130
Validation auc = 0.687793
Epoch 131
Validation auc = 0.688263
Epoch 132
Validation auc = 0.697653
Epoch 133
Validation auc = 0.707981
Epoch 134
Loss = 4.4260e-02, PNorm = 88.7545, GNorm = 3.2247, lr_0 = 8.0476e-04
Validation auc = 0.714085
Epoch 135
Validation auc = 0.720188
Epoch 136
Validation auc = 0.707042
Epoch 137
Validation auc = 0.691080
Epoch 138
Validation auc = 0.686385
Epoch 139
Loss = 4.0516e-02, PNorm = 88.8666, GNorm = 2.1304, lr_0 = 7.9788e-04
Validation auc = 0.687793
Epoch 140
Validation auc = 0.680282
Epoch 141
Validation auc = 0.676526
Epoch 142
Validation auc = 0.669953
Epoch 143
Validation auc = 0.673239
Epoch 144
Loss = 5.6560e-02, PNorm = 89.0196, GNorm = 0.8661, lr_0 = 7.9106e-04
Validation auc = 0.689202
Epoch 145
Validation auc = 0.706573
Epoch 146
Validation auc = 0.721127
Epoch 147
Validation auc = 0.722066
Epoch 148
Validation auc = 0.725352
Epoch 149
Loss = 4.7275e-02, PNorm = 89.1178, GNorm = 2.6119, lr_0 = 7.8430e-04
Validation auc = 0.709390
Epoch 150
Validation auc = 0.688263
Epoch 151
Validation auc = 0.674648
Epoch 152
Validation auc = 0.669953
Epoch 153
Validation auc = 0.670423
Epoch 154
Loss = 1.0279e-01, PNorm = 89.3055, GNorm = 2.4232, lr_0 = 7.7759e-04
Validation auc = 0.676056
Epoch 155
Validation auc = 0.689671
Epoch 156
Validation auc = 0.690610
Epoch 157
Validation auc = 0.693427
Epoch 158
Validation auc = 0.700939
Epoch 159
Loss = 6.8104e-03, PNorm = 89.4536, GNorm = 0.3024, lr_0 = 7.7095e-04
Validation auc = 0.705164
Epoch 160
Validation auc = 0.707981
Epoch 161
Validation auc = 0.700000
Epoch 162
Validation auc = 0.703286
Epoch 163
Validation auc = 0.707042
Epoch 164
Loss = 8.9666e-03, PNorm = 89.5338, GNorm = 0.4753, lr_0 = 7.6436e-04
Validation auc = 0.709390
Epoch 165
Validation auc = 0.705164
Epoch 166
Validation auc = 0.702817
Epoch 167
Validation auc = 0.707042
Epoch 168
Validation auc = 0.688263
Epoch 169
Loss = 2.2944e-02, PNorm = 89.6724, GNorm = 1.6418, lr_0 = 7.5782e-04
Validation auc = 0.677934
Epoch 170
Validation auc = 0.675117
Epoch 171
Validation auc = 0.672300
Epoch 172
Validation auc = 0.676056
Epoch 173
Validation auc = 0.679343
Epoch 174
Loss = 2.1286e-02, PNorm = 89.8014, GNorm = 1.3280, lr_0 = 7.5134e-04
Validation auc = 0.678873
Epoch 175
Validation auc = 0.682160
Epoch 176
Validation auc = 0.690141
Epoch 177
Validation auc = 0.695775
Epoch 178
Validation auc = 0.701408
Epoch 179
Loss = 5.9917e-02, PNorm = 89.9266, GNorm = 5.5284, lr_0 = 7.4492e-04
Validation auc = 0.702817
Epoch 180
Validation auc = 0.702347
Epoch 181
Validation auc = 0.705164
Epoch 182
Validation auc = 0.712207
Epoch 183
Validation auc = 0.705634
Epoch 184
Loss = 3.4744e-02, PNorm = 90.0163, GNorm = 1.5806, lr_0 = 7.3855e-04
Validation auc = 0.709390
Epoch 185
Validation auc = 0.709390
Epoch 186
Validation auc = 0.704225
Epoch 187
Validation auc = 0.702347
Epoch 188
Validation auc = 0.696244
Epoch 189
Loss = 3.3591e-02, PNorm = 90.0801, GNorm = 0.3256, lr_0 = 7.3224e-04
Validation auc = 0.693427
Epoch 190
Validation auc = 0.691080
Epoch 191
Validation auc = 0.692488
Epoch 192
Validation auc = 0.691549
Epoch 193
Validation auc = 0.696714
Epoch 194
Loss = 4.1603e-03, PNorm = 90.1357, GNorm = 0.1278, lr_0 = 7.2598e-04
Validation auc = 0.706103
Epoch 195
Validation auc = 0.708451
Epoch 196
Validation auc = 0.706103
Epoch 197
Validation auc = 0.701408
Epoch 198
Validation auc = 0.696714
Epoch 199
Loss = 2.5047e-02, PNorm = 90.1993, GNorm = 0.1105, lr_0 = 7.1977e-04
Validation auc = 0.686385
Epoch 200
Validation auc = 0.681690
Epoch 201
Validation auc = 0.678404
Epoch 202
Validation auc = 0.675587
Epoch 203
Validation auc = 0.678873
Epoch 204
Loss = 3.0134e-03, PNorm = 90.2544, GNorm = 0.0645, lr_0 = 7.1362e-04
Validation auc = 0.679812
Epoch 205
Validation auc = 0.683099
Epoch 206
Validation auc = 0.682160
Epoch 207
Validation auc = 0.689202
Epoch 208
Validation auc = 0.689671
Epoch 209
Loss = 2.1940e-03, PNorm = 90.2894, GNorm = 0.0939, lr_0 = 7.0752e-04
Validation auc = 0.687793
Epoch 210
Validation auc = 0.689671
Epoch 211
Validation auc = 0.693427
Epoch 212
Validation auc = 0.690610
Epoch 213
Validation auc = 0.692488
Epoch 214
Loss = 2.5474e-03, PNorm = 90.3174, GNorm = 0.0620, lr_0 = 7.0147e-04
Validation auc = 0.694366
Epoch 215
Validation auc = 0.696244
Epoch 216
Validation auc = 0.695775
Epoch 217
Validation auc = 0.691080
Epoch 218
Validation auc = 0.685915
Epoch 219
Loss = 9.8585e-03, PNorm = 90.3485, GNorm = 1.1454, lr_0 = 6.9548e-04
Validation auc = 0.685446
Epoch 220
Validation auc = 0.678404
Epoch 221
Validation auc = 0.683099
Epoch 222
Validation auc = 0.683099
Epoch 223
Validation auc = 0.692488
Epoch 224
Loss = 2.8291e-02, PNorm = 90.4057, GNorm = 3.1350, lr_0 = 6.8953e-04
Validation auc = 0.692958
Epoch 225
Validation auc = 0.699061
Epoch 226
Validation auc = 0.702817
Epoch 227
Validation auc = 0.716901
Epoch 228
Validation auc = 0.713615
Epoch 229
Loss = 6.5499e-03, PNorm = 90.4665, GNorm = 0.6287, lr_0 = 6.8364e-04
Validation auc = 0.708451
Epoch 230
Validation auc = 0.698592
Epoch 231
Validation auc = 0.692019
Epoch 232
Validation auc = 0.686854
Epoch 233
Validation auc = 0.682629
Epoch 234
Loss = 2.0325e-02, PNorm = 90.5281, GNorm = 0.5780, lr_0 = 6.7779e-04
Validation auc = 0.692019
Epoch 235
Validation auc = 0.700000
Epoch 236
Validation auc = 0.705164
Epoch 237
Validation auc = 0.715493
Epoch 238
Validation auc = 0.723944
Epoch 239
Loss = 4.8612e-03, PNorm = 90.5813, GNorm = 0.1612, lr_0 = 6.7200e-04
Validation auc = 0.733333
Epoch 240
Validation auc = 0.730986
Epoch 241
Validation auc = 0.723005
Epoch 242
Validation auc = 0.717371
Epoch 243
Validation auc = 0.708451
Epoch 244
Loss = 4.0814e-03, PNorm = 90.6418, GNorm = 0.7603, lr_0 = 6.6625e-04
Validation auc = 0.703756
Epoch 245
Validation auc = 0.701878
Epoch 246
Validation auc = 0.701408
Epoch 247
Validation auc = 0.700939
Epoch 248
Validation auc = 0.700000
Epoch 249
Loss = 6.6102e-04, PNorm = 90.6782, GNorm = 0.0278, lr_0 = 6.6056e-04
Validation auc = 0.696244
Epoch 250
Validation auc = 0.695775
Epoch 251
Validation auc = 0.694836
Epoch 252
Validation auc = 0.693897
Epoch 253
Validation auc = 0.696714
Epoch 254
Loss = 4.0402e-03, PNorm = 90.7037, GNorm = 0.4739, lr_0 = 6.5491e-04
Validation auc = 0.698592
Epoch 255
Validation auc = 0.700469
Epoch 256
Validation auc = 0.701878
Epoch 257
Validation auc = 0.699061
Epoch 258
Validation auc = 0.697183
Epoch 259
Loss = 6.7492e-03, PNorm = 90.7372, GNorm = 0.1457, lr_0 = 6.4931e-04
Validation auc = 0.690141
Epoch 260
Validation auc = 0.693897
Epoch 261
Validation auc = 0.701408
Epoch 262
Validation auc = 0.707981
Epoch 263
Validation auc = 0.710798
Epoch 264
Loss = 3.5923e-02, PNorm = 90.7880, GNorm = 0.0840, lr_0 = 6.4376e-04
Validation auc = 0.713146
Epoch 265
Validation auc = 0.711268
Epoch 266
Validation auc = 0.705164
Epoch 267
Validation auc = 0.704225
Epoch 268
Validation auc = 0.701408
Epoch 269
Loss = 1.3840e-02, PNorm = 90.8609, GNorm = 0.4271, lr_0 = 6.3826e-04
Validation auc = 0.701408
Epoch 270
Validation auc = 0.703286
Epoch 271
Validation auc = 0.708920
Epoch 272
Validation auc = 0.714085
Epoch 273
Validation auc = 0.715493
Epoch 274
Loss = 3.1301e-03, PNorm = 90.9105, GNorm = 0.4703, lr_0 = 6.3280e-04
Validation auc = 0.718779
Epoch 275
Validation auc = 0.718779
Epoch 276
Validation auc = 0.715962
Epoch 277
Validation auc = 0.715023
Epoch 278
Validation auc = 0.712207
Epoch 279
Loss = 1.3735e-03, PNorm = 90.9750, GNorm = 0.1004, lr_0 = 6.2739e-04
Validation auc = 0.713146
Epoch 280
Validation auc = 0.711268
Epoch 281
Validation auc = 0.716901
Epoch 282
Validation auc = 0.719718
Epoch 283
Validation auc = 0.723474
Epoch 284
Loss = 1.7440e-03, PNorm = 91.0009, GNorm = 0.0479, lr_0 = 6.2203e-04
Validation auc = 0.723005
Epoch 285
Validation auc = 0.723474
Epoch 286
Validation auc = 0.724413
Epoch 287
Validation auc = 0.711737
Epoch 288
Validation auc = 0.702347
Epoch 289
Loss = 6.1388e-02, PNorm = 91.0501, GNorm = 2.6499, lr_0 = 6.1671e-04
Validation auc = 0.698592
Epoch 290
Validation auc = 0.684977
Epoch 291
Validation auc = 0.679812
Epoch 292
Validation auc = 0.677465
Epoch 293
Validation auc = 0.676056
Epoch 294
Loss = 3.9185e-02, PNorm = 91.0728, GNorm = 0.3812, lr_0 = 6.1144e-04
Validation auc = 0.676526
Epoch 295
Validation auc = 0.673239
Epoch 296
Validation auc = 0.675587
Epoch 297
Validation auc = 0.675117
Epoch 298
Validation auc = 0.676995
Epoch 299
Loss = 3.0448e-03, PNorm = 91.1312, GNorm = 0.1492, lr_0 = 6.0621e-04
Validation auc = 0.677934
Model 0 best validation auc = 0.830986 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.839253
Ensemble test auc = 0.839253
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.662441
Epoch 1
Validation auc = 0.710798
Epoch 2
Validation auc = 0.720188
Epoch 3
Validation auc = 0.728638
Epoch 4
Loss = 8.1193e-01, PNorm = 86.2066, GNorm = 5.7851, lr_0 = 6.5000e-04
Validation auc = 0.717371
Epoch 5
Validation auc = 0.716901
Epoch 6
Validation auc = 0.700469
Epoch 7
Validation auc = 0.724413
Epoch 8
Validation auc = 0.736620
Epoch 9
Loss = 3.3304e-01, PNorm = 86.5439, GNorm = 10.7963, lr_0 = 9.9743e-04
Validation auc = 0.729577
Epoch 10
Validation auc = 0.724883
Epoch 11
Validation auc = 0.711737
Epoch 12
Validation auc = 0.701408
Epoch 13
Validation auc = 0.707512
Epoch 14
Loss = 4.7761e-01, PNorm = 86.8707, GNorm = 8.1231, lr_0 = 9.8890e-04
Validation auc = 0.707981
Epoch 15
Validation auc = 0.707981
Epoch 16
Validation auc = 0.708920
Epoch 17
Validation auc = 0.715023
Epoch 18
Validation auc = 0.716901
Epoch 19
Loss = 1.7829e-01, PNorm = 87.0727, GNorm = 5.3036, lr_0 = 9.8045e-04
Validation auc = 0.719249
Epoch 20
Validation auc = 0.710798
Epoch 21
Validation auc = 0.707512
Epoch 22
Validation auc = 0.707512
Epoch 23
Validation auc = 0.711737
Epoch 24
Loss = 7.3208e-02, PNorm = 87.1960, GNorm = 1.9379, lr_0 = 9.7207e-04
Validation auc = 0.704225
Epoch 25
Validation auc = 0.709859
Epoch 26
Validation auc = 0.719249
Epoch 27
Validation auc = 0.712676
Epoch 28
Validation auc = 0.689671
Epoch 29
Loss = 1.0253e-01, PNorm = 87.2737, GNorm = 3.9147, lr_0 = 9.6376e-04
Validation auc = 0.682160
Epoch 30
Validation auc = 0.675587
Epoch 31
Validation auc = 0.673709
Epoch 32
Validation auc = 0.672300
Epoch 33
Validation auc = 0.689202
Epoch 34
Loss = 9.9574e-02, PNorm = 87.3445, GNorm = 2.6264, lr_0 = 9.5552e-04
Validation auc = 0.694836
Epoch 35
Validation auc = 0.694836
Epoch 36
Validation auc = 0.682160
Epoch 37
Validation auc = 0.671362
Epoch 38
Validation auc = 0.677934
Epoch 39
Loss = 4.1143e-02, PNorm = 87.4116, GNorm = 1.0731, lr_0 = 9.4735e-04
Validation auc = 0.681690
Epoch 40
Validation auc = 0.683568
Epoch 41
Validation auc = 0.693427
Epoch 42
Validation auc = 0.702817
Epoch 43
Validation auc = 0.713615
Epoch 44
Loss = 9.1388e-02, PNorm = 87.4782, GNorm = 3.3431, lr_0 = 9.3925e-04
Validation auc = 0.718779
Epoch 45
Validation auc = 0.712676
Epoch 46
Validation auc = 0.706573
Epoch 47
Validation auc = 0.692488
Epoch 48
Validation auc = 0.672770
Epoch 49
Loss = 1.0234e-01, PNorm = 87.5368, GNorm = 2.5238, lr_0 = 9.3122e-04
Validation auc = 0.673239
Epoch 50
Validation auc = 0.687324
Epoch 51
Validation auc = 0.690141
Epoch 52
Validation auc = 0.693427
Epoch 53
Validation auc = 0.690610
Epoch 54
Loss = 8.7083e-02, PNorm = 87.5793, GNorm = 2.2279, lr_0 = 9.2326e-04
Validation auc = 0.698122
Epoch 55
Validation auc = 0.704695
Epoch 56
Validation auc = 0.704225
Epoch 57
Validation auc = 0.709859
Epoch 58
Validation auc = 0.711268
Epoch 59
Loss = 5.1114e-02, PNorm = 87.6566, GNorm = 2.2805, lr_0 = 9.1537e-04
Validation auc = 0.712676
Epoch 60
Validation auc = 0.707042
Epoch 61
Validation auc = 0.701878
Epoch 62
Validation auc = 0.691080
Epoch 63
Validation auc = 0.683099
Epoch 64
Loss = 1.8045e-02, PNorm = 87.7274, GNorm = 0.5342, lr_0 = 9.0754e-04
Validation auc = 0.686854
Epoch 65
Validation auc = 0.680751
Epoch 66
Validation auc = 0.681690
Epoch 67
Validation auc = 0.675587
Epoch 68
Validation auc = 0.665258
Epoch 69
Loss = 6.1024e-02, PNorm = 87.7895, GNorm = 1.6304, lr_0 = 8.9978e-04
Validation auc = 0.671362
Epoch 70
Validation auc = 0.661502
Epoch 71
Validation auc = 0.658216
Epoch 72
Validation auc = 0.663380
Epoch 73
Validation auc = 0.662911
Epoch 74
Loss = 6.8597e-02, PNorm = 87.8555, GNorm = 0.4816, lr_0 = 8.9209e-04
Validation auc = 0.658216
Epoch 75
Validation auc = 0.663850
Epoch 76
Validation auc = 0.674648
Epoch 77
Validation auc = 0.686854
Epoch 78
Validation auc = 0.696714
Epoch 79
Loss = 5.0965e-02, PNorm = 87.9479, GNorm = 3.4730, lr_0 = 8.8447e-04
Validation auc = 0.698592
Epoch 80
Validation auc = 0.694366
Epoch 81
Validation auc = 0.690141
Epoch 82
Validation auc = 0.685915
Epoch 83
Validation auc = 0.687793
Epoch 84
Loss = 9.6056e-02, PNorm = 88.0204, GNorm = 1.8225, lr_0 = 8.7691e-04
Validation auc = 0.690610
Epoch 85
Validation auc = 0.677465
Epoch 86
Validation auc = 0.660563
Epoch 87
Validation auc = 0.653521
Epoch 88
Validation auc = 0.652113
Epoch 89
Loss = 1.8165e-02, PNorm = 88.0722, GNorm = 0.2807, lr_0 = 8.6941e-04
Validation auc = 0.666197
Epoch 90
Validation auc = 0.674648
Epoch 91
Validation auc = 0.670892
Epoch 92
Validation auc = 0.650704
Epoch 93
Validation auc = 0.649296
Epoch 94
Loss = 3.7527e-02, PNorm = 88.1225, GNorm = 2.5453, lr_0 = 8.6198e-04
Validation auc = 0.651174
Epoch 95
Validation auc = 0.656808
Epoch 96
Validation auc = 0.669484
Epoch 97
Validation auc = 0.676526
Epoch 98
Validation auc = 0.676056
Epoch 99
Loss = 1.3650e-02, PNorm = 88.1965, GNorm = 0.4821, lr_0 = 8.5461e-04
Validation auc = 0.680751
Epoch 100
Validation auc = 0.690610
Epoch 101
Validation auc = 0.696714
Epoch 102
Validation auc = 0.701878
Epoch 103
Validation auc = 0.705634
Epoch 104
Loss = 1.6757e-02, PNorm = 88.2358, GNorm = 1.3637, lr_0 = 8.4730e-04
Validation auc = 0.700000
Epoch 105
Validation auc = 0.703756
Epoch 106
Validation auc = 0.705164
Epoch 107
Validation auc = 0.709390
Epoch 108
Validation auc = 0.702817
Epoch 109
Loss = 1.9745e-02, PNorm = 88.3186, GNorm = 0.7791, lr_0 = 8.4006e-04
Validation auc = 0.692958
Epoch 110
Validation auc = 0.682160
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.659155
Epoch 113
Validation auc = 0.657277
Epoch 114
Loss = 5.6966e-02, PNorm = 88.3786, GNorm = 1.4835, lr_0 = 8.3288e-04
Validation auc = 0.667136
Epoch 115
Validation auc = 0.676995
Epoch 116
Validation auc = 0.683568
Epoch 117
Validation auc = 0.686385
Epoch 118
Validation auc = 0.688263
Epoch 119
Loss = 2.5929e-02, PNorm = 88.4289, GNorm = 1.9974, lr_0 = 8.2576e-04
Validation auc = 0.690610
Epoch 120
Validation auc = 0.694836
Epoch 121
Validation auc = 0.697183
Epoch 122
Validation auc = 0.689671
Epoch 123
Validation auc = 0.676995
Epoch 124
Loss = 2.5717e-02, PNorm = 88.4676, GNorm = 0.4714, lr_0 = 8.1870e-04
Validation auc = 0.669014
Epoch 125
Validation auc = 0.656338
Epoch 126
Validation auc = 0.656338
Epoch 127
Validation auc = 0.663380
Epoch 128
Validation auc = 0.665258
Epoch 129
Loss = 8.2988e-02, PNorm = 88.5075, GNorm = 1.5769, lr_0 = 8.1170e-04
Validation auc = 0.663850
Epoch 130
Validation auc = 0.668545
Epoch 131
Validation auc = 0.658685
Epoch 132
Validation auc = 0.653052
Epoch 133
Validation auc = 0.644131
Epoch 134
Loss = 1.3084e-02, PNorm = 88.5999, GNorm = 0.4350, lr_0 = 8.0476e-04
Validation auc = 0.630047
Epoch 135
Validation auc = 0.624883
Epoch 136
Validation auc = 0.623005
Epoch 137
Validation auc = 0.624413
Epoch 138
Validation auc = 0.629108
Epoch 139
Loss = 4.9272e-03, PNorm = 88.6602, GNorm = 0.0269, lr_0 = 7.9788e-04
Validation auc = 0.633803
Epoch 140
Validation auc = 0.635211
Epoch 141
Validation auc = 0.647418
Epoch 142
Validation auc = 0.660563
Epoch 143
Validation auc = 0.664319
Epoch 144
Loss = 6.2761e-03, PNorm = 88.7125, GNorm = 0.2348, lr_0 = 7.9106e-04
Validation auc = 0.673709
Epoch 145
Validation auc = 0.676056
Epoch 146
Validation auc = 0.674178
Epoch 147
Validation auc = 0.664319
Epoch 148
Validation auc = 0.660563
Epoch 149
Loss = 2.0056e-02, PNorm = 88.7369, GNorm = 0.4716, lr_0 = 7.8430e-04
Validation auc = 0.654460
Epoch 150
Validation auc = 0.651174
Epoch 151
Validation auc = 0.649296
Epoch 152
Validation auc = 0.648357
Epoch 153
Validation auc = 0.651174
Epoch 154
Loss = 9.2689e-03, PNorm = 88.8322, GNorm = 0.4552, lr_0 = 7.7759e-04
Validation auc = 0.653521
Epoch 155
Validation auc = 0.655869
Epoch 156
Validation auc = 0.654460
Epoch 157
Validation auc = 0.652113
Epoch 158
Validation auc = 0.652113
Epoch 159
Loss = 2.2063e-02, PNorm = 88.9067, GNorm = 3.1567, lr_0 = 7.7095e-04
Validation auc = 0.652582
Epoch 160
Validation auc = 0.654930
Epoch 161
Validation auc = 0.654930
Epoch 162
Validation auc = 0.648357
Epoch 163
Validation auc = 0.646948
Epoch 164
Loss = 4.0324e-03, PNorm = 88.9486, GNorm = 0.1165, lr_0 = 7.6436e-04
Validation auc = 0.646479
Epoch 165
Validation auc = 0.647887
Epoch 166
Validation auc = 0.649296
Epoch 167
Validation auc = 0.651643
Epoch 168
Validation auc = 0.651643
Epoch 169
Loss = 5.6850e-03, PNorm = 88.9711, GNorm = 0.4046, lr_0 = 7.5782e-04
Validation auc = 0.652113
Epoch 170
Validation auc = 0.654460
Epoch 171
Validation auc = 0.656808
Epoch 172
Validation auc = 0.661502
Epoch 173
Validation auc = 0.662441
Epoch 174
Loss = 2.1714e-03, PNorm = 88.9857, GNorm = 0.1415, lr_0 = 7.5134e-04
Validation auc = 0.662441
Epoch 175
Validation auc = 0.659155
Epoch 176
Validation auc = 0.652113
Epoch 177
Validation auc = 0.649296
Epoch 178
Validation auc = 0.646948
Epoch 179
Loss = 1.5368e-02, PNorm = 89.0016, GNorm = 1.0959, lr_0 = 7.4492e-04
Validation auc = 0.643192
Epoch 180
Validation auc = 0.647418
Epoch 181
Validation auc = 0.645070
Epoch 182
Validation auc = 0.649061
Epoch 183
Validation auc = 0.653521
Epoch 184
Loss = 1.1664e-01, PNorm = 89.0212, GNorm = 6.6647, lr_0 = 7.3855e-04
Validation auc = 0.655869
Epoch 185
Validation auc = 0.663380
Epoch 186
Validation auc = 0.664789
Epoch 187
Validation auc = 0.645540
Epoch 188
Validation auc = 0.632394
Epoch 189
Loss = 6.6635e-02, PNorm = 89.0741, GNorm = 4.6891, lr_0 = 7.3224e-04
Validation auc = 0.638028
Epoch 190
Validation auc = 0.655869
Epoch 191
Validation auc = 0.657746
Epoch 192
Validation auc = 0.648357
Epoch 193
Validation auc = 0.638028
Epoch 194
Loss = 1.1873e-02, PNorm = 89.1460, GNorm = 1.0380, lr_0 = 7.2598e-04
Validation auc = 0.634272
Epoch 195
Validation auc = 0.633333
Epoch 196
Validation auc = 0.639906
Epoch 197
Validation auc = 0.644601
Epoch 198
Validation auc = 0.651643
Epoch 199
Loss = 3.4445e-02, PNorm = 89.2243, GNorm = 3.1596, lr_0 = 7.1977e-04
Validation auc = 0.652582
Epoch 200
Validation auc = 0.668545
Epoch 201
Validation auc = 0.688263
Epoch 202
Validation auc = 0.699061
Epoch 203
Validation auc = 0.698122
Epoch 204
Loss = 3.0125e-02, PNorm = 89.3156, GNorm = 1.7721, lr_0 = 7.1362e-04
Validation auc = 0.692019
Epoch 205
Validation auc = 0.681690
Epoch 206
Validation auc = 0.688732
Epoch 207
Validation auc = 0.684977
Epoch 208
Validation auc = 0.675117
Epoch 209
Loss = 4.9670e-03, PNorm = 89.4092, GNorm = 0.2761, lr_0 = 7.0752e-04
Validation auc = 0.669014
Epoch 210
Validation auc = 0.660563
Epoch 211
Validation auc = 0.667136
Epoch 212
Validation auc = 0.668075
Epoch 213
Validation auc = 0.666197
Epoch 214
Loss = 4.1841e-02, PNorm = 89.5105, GNorm = 2.2219, lr_0 = 7.0147e-04
Validation auc = 0.668545
Epoch 215
Validation auc = 0.669484
Epoch 216
Validation auc = 0.674648
Epoch 217
Validation auc = 0.679343
Epoch 218
Validation auc = 0.680751
Epoch 219
Loss = 1.2146e-02, PNorm = 89.6343, GNorm = 0.6875, lr_0 = 6.9548e-04
Validation auc = 0.680282
Epoch 220
Validation auc = 0.675117
Epoch 221
Validation auc = 0.671362
Epoch 222
Validation auc = 0.663380
Epoch 223
Validation auc = 0.660563
Epoch 224
Loss = 4.2681e-02, PNorm = 89.7140, GNorm = 2.3908, lr_0 = 6.8953e-04
Validation auc = 0.659155
Epoch 225
Validation auc = 0.656338
Epoch 226
Validation auc = 0.661033
Epoch 227
Validation auc = 0.650235
Epoch 228
Validation auc = 0.648357
Epoch 229
Loss = 4.7241e-03, PNorm = 89.8151, GNorm = 0.2714, lr_0 = 6.8364e-04
Validation auc = 0.648826
Epoch 230
Validation auc = 0.648826
Epoch 231
Validation auc = 0.662441
Epoch 232
Validation auc = 0.676056
Epoch 233
Validation auc = 0.678873
Epoch 234
Loss = 9.6413e-03, PNorm = 89.8878, GNorm = 0.5826, lr_0 = 6.7779e-04
Validation auc = 0.679343
Epoch 235
Validation auc = 0.675587
Epoch 236
Validation auc = 0.668075
Epoch 237
Validation auc = 0.663380
Epoch 238
Validation auc = 0.665728
Epoch 239
Loss = 8.8777e-03, PNorm = 89.9221, GNorm = 0.4895, lr_0 = 6.7200e-04
Validation auc = 0.673239
Epoch 240
Validation auc = 0.676526
Epoch 241
Validation auc = 0.671362
Epoch 242
Validation auc = 0.668545
Epoch 243
Validation auc = 0.671831
Epoch 244
Loss = 1.4179e-02, PNorm = 89.9529, GNorm = 0.6877, lr_0 = 6.6625e-04
Validation auc = 0.666197
Epoch 245
Validation auc = 0.665258
Epoch 246
Validation auc = 0.669014
Epoch 247
Validation auc = 0.668075
Epoch 248
Validation auc = 0.664789
Epoch 249
Loss = 8.1112e-03, PNorm = 89.9878, GNorm = 0.4487, lr_0 = 6.6056e-04
Validation auc = 0.662911
Epoch 250
Validation auc = 0.655869
Epoch 251
Validation auc = 0.658685
Epoch 252
Validation auc = 0.659624
Epoch 253
Validation auc = 0.661033
Epoch 254
Loss = 1.4159e-02, PNorm = 90.0166, GNorm = 2.3753, lr_0 = 6.5491e-04
Validation auc = 0.663850
Epoch 255
Validation auc = 0.662911
Epoch 256
Validation auc = 0.666667
Epoch 257
Validation auc = 0.671362
Epoch 258
Validation auc = 0.677465
Epoch 259
Loss = 1.6331e-02, PNorm = 90.0378, GNorm = 0.2231, lr_0 = 6.4931e-04
Validation auc = 0.683568
Epoch 260
Validation auc = 0.676995
Epoch 261
Validation auc = 0.674648
Epoch 262
Validation auc = 0.671362
Epoch 263
Validation auc = 0.673239
Epoch 264
Loss = 2.5395e-03, PNorm = 90.0582, GNorm = 0.1305, lr_0 = 6.4376e-04
Validation auc = 0.672300
Epoch 265
Validation auc = 0.668545
Epoch 266
Validation auc = 0.665728
Epoch 267
Validation auc = 0.663380
Epoch 268
Validation auc = 0.663850
Epoch 269
Loss = 6.0391e-03, PNorm = 90.0701, GNorm = 0.1266, lr_0 = 6.3826e-04
Validation auc = 0.666197
Epoch 270
Validation auc = 0.666667
Epoch 271
Validation auc = 0.672770
Epoch 272
Validation auc = 0.679343
Epoch 273
Validation auc = 0.679812
Epoch 274
Loss = 3.1889e-03, PNorm = 90.0834, GNorm = 0.1519, lr_0 = 6.3280e-04
Validation auc = 0.678873
Epoch 275
Validation auc = 0.679343
Epoch 276
Validation auc = 0.679812
Epoch 277
Validation auc = 0.680282
Epoch 278
Validation auc = 0.677934
Epoch 279
Loss = 2.6414e-03, PNorm = 90.0956, GNorm = 0.0496, lr_0 = 6.2739e-04
Validation auc = 0.673709
Epoch 280
Validation auc = 0.670423
Epoch 281
Validation auc = 0.668545
Epoch 282
Validation auc = 0.665258
Epoch 283
Validation auc = 0.659624
Epoch 284
Loss = 1.0579e-02, PNorm = 90.1049, GNorm = 0.1569, lr_0 = 6.2203e-04
Validation auc = 0.658216
Epoch 285
Validation auc = 0.658216
Epoch 286
Validation auc = 0.660563
Epoch 287
Validation auc = 0.662911
Epoch 288
Validation auc = 0.664789
Epoch 289
Loss = 3.1891e-03, PNorm = 90.1147, GNorm = 0.0341, lr_0 = 6.1671e-04
Validation auc = 0.663850
Epoch 290
Validation auc = 0.663850
Epoch 291
Validation auc = 0.660563
Epoch 292
Validation auc = 0.666667
Epoch 293
Validation auc = 0.666197
Epoch 294
Loss = 1.7191e-03, PNorm = 90.1311, GNorm = 0.0675, lr_0 = 6.1144e-04
Validation auc = 0.669484
Epoch 295
Validation auc = 0.669953
Epoch 296
Validation auc = 0.667606
Epoch 297
Validation auc = 0.669484
Epoch 298
Validation auc = 0.669953
Epoch 299
Loss = 1.8721e-03, PNorm = 90.1441, GNorm = 0.0488, lr_0 = 6.0621e-04
Validation auc = 0.672770
Model 0 best validation auc = 0.736620 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.820590
Ensemble test auc = 0.820590
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.703286
Epoch 1
Validation auc = 0.723005
Epoch 2
Validation auc = 0.744601
Epoch 3
Validation auc = 0.767606
Epoch 4
Loss = 8.3401e-01, PNorm = 86.2016, GNorm = 13.6479, lr_0 = 6.5000e-04
Validation auc = 0.755869
Epoch 5
Validation auc = 0.736620
Epoch 6
Validation auc = 0.661972
Epoch 7
Validation auc = 0.664789
Epoch 8
Validation auc = 0.723005
Epoch 9
Loss = 5.4341e-01, PNorm = 86.5223, GNorm = 5.6372, lr_0 = 9.9743e-04
Validation auc = 0.752113
Epoch 10
Validation auc = 0.719718
Epoch 11
Validation auc = 0.730986
Epoch 12
Validation auc = 0.708451
Epoch 13
Validation auc = 0.707042
Epoch 14
Loss = 5.2733e-01, PNorm = 86.8388, GNorm = 9.0887, lr_0 = 9.8890e-04
Validation auc = 0.715493
Epoch 15
Validation auc = 0.734272
Epoch 16
Validation auc = 0.737559
Epoch 17
Validation auc = 0.734272
Epoch 18
Validation auc = 0.734272
Epoch 19
Loss = 1.2457e-01, PNorm = 87.0351, GNorm = 2.3294, lr_0 = 9.8045e-04
Validation auc = 0.744131
Epoch 20
Validation auc = 0.747887
Epoch 21
Validation auc = 0.744601
Epoch 22
Validation auc = 0.746009
Epoch 23
Validation auc = 0.724883
Epoch 24
Loss = 2.0149e-01, PNorm = 87.1533, GNorm = 3.5103, lr_0 = 9.7207e-04
Validation auc = 0.700939
Epoch 25
Validation auc = 0.721596
Epoch 26
Validation auc = 0.737089
Epoch 27
Validation auc = 0.738967
Epoch 28
Validation auc = 0.721596
Epoch 29
Loss = 1.4490e-01, PNorm = 87.2365, GNorm = 1.6658, lr_0 = 9.6376e-04
Validation auc = 0.703756
Epoch 30
Validation auc = 0.689671
Epoch 31
Validation auc = 0.691080
Epoch 32
Validation auc = 0.692019
Epoch 33
Validation auc = 0.688732
Epoch 34
Loss = 1.6806e-01, PNorm = 87.3078, GNorm = 3.1543, lr_0 = 9.5552e-04
Validation auc = 0.688732
Epoch 35
Validation auc = 0.705634
Epoch 36
Validation auc = 0.721127
Epoch 37
Validation auc = 0.728638
Epoch 38
Validation auc = 0.742254
Epoch 39
Loss = 1.2327e-01, PNorm = 87.3806, GNorm = 3.1142, lr_0 = 9.4735e-04
Validation auc = 0.723944
Epoch 40
Validation auc = 0.713615
Epoch 41
Validation auc = 0.704695
Epoch 42
Validation auc = 0.694836
Epoch 43
Validation auc = 0.684038
Epoch 44
Loss = 8.8499e-02, PNorm = 87.4499, GNorm = 1.2603, lr_0 = 9.3925e-04
Validation auc = 0.674178
Epoch 45
Validation auc = 0.690141
Epoch 46
Validation auc = 0.692488
Epoch 47
Validation auc = 0.686854
Epoch 48
Validation auc = 0.689202
Epoch 49
Loss = 1.0608e-01, PNorm = 87.5237, GNorm = 1.0289, lr_0 = 9.3122e-04
Validation auc = 0.692488
Epoch 50
Validation auc = 0.684977
Epoch 51
Validation auc = 0.672770
Epoch 52
Validation auc = 0.664319
Epoch 53
Validation auc = 0.663380
Epoch 54
Loss = 8.6233e-02, PNorm = 87.5910, GNorm = 2.9827, lr_0 = 9.2326e-04
Validation auc = 0.675117
Epoch 55
Validation auc = 0.688732
Epoch 56
Validation auc = 0.688263
Epoch 57
Validation auc = 0.680282
Epoch 58
Validation auc = 0.677934
Epoch 59
Loss = 5.7840e-02, PNorm = 87.6856, GNorm = 0.7918, lr_0 = 9.1537e-04
Validation auc = 0.672300
Epoch 60
Validation auc = 0.661972
Epoch 61
Validation auc = 0.656338
Epoch 62
Validation auc = 0.653521
Epoch 63
Validation auc = 0.644601
Epoch 64
Loss = 5.3344e-02, PNorm = 87.8343, GNorm = 0.4498, lr_0 = 9.0754e-04
Validation auc = 0.651174
Epoch 65
Validation auc = 0.666197
Epoch 66
Validation auc = 0.700469
Epoch 67
Validation auc = 0.719718
Epoch 68
Validation auc = 0.735681
Epoch 69
Loss = 2.5361e-01, PNorm = 87.9546, GNorm = 2.5915, lr_0 = 8.9978e-04
Validation auc = 0.717840
Epoch 70
Validation auc = 0.707042
Epoch 71
Validation auc = 0.697653
Epoch 72
Validation auc = 0.686385
Epoch 73
Validation auc = 0.670892
Epoch 74
Loss = 5.1730e-02, PNorm = 88.1459, GNorm = 0.6172, lr_0 = 8.9209e-04
Validation auc = 0.665728
Epoch 75
Validation auc = 0.669014
Epoch 76
Validation auc = 0.675117
Epoch 77
Validation auc = 0.678404
Epoch 78
Validation auc = 0.680751
Epoch 79
Loss = 4.8817e-02, PNorm = 88.2833, GNorm = 1.8426, lr_0 = 8.8447e-04
Validation auc = 0.687324
Epoch 80
Validation auc = 0.685915
Epoch 81
Validation auc = 0.692019
Epoch 82
Validation auc = 0.701408
Epoch 83
Validation auc = 0.700000
Epoch 84
Loss = 2.1881e-02, PNorm = 88.3819, GNorm = 2.0738, lr_0 = 8.7691e-04
Validation auc = 0.704695
Epoch 85
Validation auc = 0.700000
Epoch 86
Validation auc = 0.698592
Epoch 87
Validation auc = 0.701878
Epoch 88
Validation auc = 0.696244
Epoch 89
Loss = 4.3727e-02, PNorm = 88.4520, GNorm = 0.2808, lr_0 = 8.6941e-04
Validation auc = 0.686385
Epoch 90
Validation auc = 0.681690
Epoch 91
Validation auc = 0.678404
Epoch 92
Validation auc = 0.679812
Epoch 93
Validation auc = 0.683568
Epoch 94
Loss = 5.3565e-02, PNorm = 88.5299, GNorm = 0.5672, lr_0 = 8.6198e-04
Validation auc = 0.685915
Epoch 95
Validation auc = 0.684038
Epoch 96
Validation auc = 0.700000
Epoch 97
Validation auc = 0.721596
Epoch 98
Validation auc = 0.730986
Epoch 99
Loss = 2.0972e-02, PNorm = 88.5961, GNorm = 0.9455, lr_0 = 8.5461e-04
Validation auc = 0.735681
Epoch 100
Validation auc = 0.730047
Epoch 101
Validation auc = 0.716432
Epoch 102
Validation auc = 0.702347
Epoch 103
Validation auc = 0.693427
Epoch 104
Loss = 2.1634e-02, PNorm = 88.6505, GNorm = 1.3549, lr_0 = 8.4730e-04
Validation auc = 0.688732
Epoch 105
Validation auc = 0.681221
Epoch 106
Validation auc = 0.677465
Epoch 107
Validation auc = 0.674178
Epoch 108
Validation auc = 0.675117
Epoch 109
Loss = 8.2851e-03, PNorm = 88.6946, GNorm = 0.7656, lr_0 = 8.4006e-04
Validation auc = 0.681690
Epoch 110
Validation auc = 0.684038
Epoch 111
Validation auc = 0.684038
Epoch 112
Validation auc = 0.688732
Epoch 113
Validation auc = 0.696714
Epoch 114
Loss = 3.7715e-02, PNorm = 88.7369, GNorm = 0.8341, lr_0 = 8.3288e-04
Validation auc = 0.700939
Epoch 115
Validation auc = 0.702817
Epoch 116
Validation auc = 0.699531
Epoch 117
Validation auc = 0.699531
Epoch 118
Validation auc = 0.696714
Epoch 119
Loss = 4.3792e-02, PNorm = 88.7768, GNorm = 1.8270, lr_0 = 8.2576e-04
Validation auc = 0.692019
Epoch 120
Validation auc = 0.692019
Epoch 121
Validation auc = 0.691549
Epoch 122
Validation auc = 0.694366
Epoch 123
Validation auc = 0.683568
Epoch 124
Loss = 4.9153e-02, PNorm = 88.8123, GNorm = 2.6153, lr_0 = 8.1870e-04
Validation auc = 0.676056
Epoch 125
Validation auc = 0.679812
Epoch 126
Validation auc = 0.685446
Epoch 127
Validation auc = 0.692019
Epoch 128
Validation auc = 0.693897
Epoch 129
Loss = 3.6120e-02, PNorm = 88.8535, GNorm = 1.6381, lr_0 = 8.1170e-04
Validation auc = 0.691080
Epoch 130
Validation auc = 0.691080
Epoch 131
Validation auc = 0.689202
Epoch 132
Validation auc = 0.686854
Epoch 133
Validation auc = 0.678404
Epoch 134
Loss = 7.7527e-03, PNorm = 88.9194, GNorm = 0.0950, lr_0 = 8.0476e-04
Validation auc = 0.669014
Epoch 135
Validation auc = 0.660094
Epoch 136
Validation auc = 0.660094
Epoch 137
Validation auc = 0.661033
Epoch 138
Validation auc = 0.659624
Epoch 139
Loss = 3.4665e-02, PNorm = 88.9806, GNorm = 0.2671, lr_0 = 7.9788e-04
Validation auc = 0.656338
Epoch 140
Validation auc = 0.660094
Epoch 141
Validation auc = 0.665258
Epoch 142
Validation auc = 0.667136
Epoch 143
Validation auc = 0.664789
Epoch 144
Loss = 1.0156e-02, PNorm = 89.0320, GNorm = 0.1762, lr_0 = 7.9106e-04
Validation auc = 0.662911
Epoch 145
Validation auc = 0.661972
Epoch 146
Validation auc = 0.661033
Epoch 147
Validation auc = 0.664789
Epoch 148
Validation auc = 0.668545
Epoch 149
Loss = 5.9801e-03, PNorm = 89.0686, GNorm = 0.7567, lr_0 = 7.8430e-04
Validation auc = 0.673709
Epoch 150
Validation auc = 0.677465
Epoch 151
Validation auc = 0.681690
Epoch 152
Validation auc = 0.682629
Epoch 153
Validation auc = 0.683568
Epoch 154
Loss = 1.8675e-03, PNorm = 89.0975, GNorm = 0.0427, lr_0 = 7.7759e-04
Validation auc = 0.685446
Epoch 155
Validation auc = 0.684038
Epoch 156
Validation auc = 0.685446
Epoch 157
Validation auc = 0.687324
Epoch 158
Validation auc = 0.691549
Epoch 159
Loss = 1.0713e-02, PNorm = 89.1194, GNorm = 0.5970, lr_0 = 7.7095e-04
Validation auc = 0.688263
Epoch 160
Validation auc = 0.691080
Epoch 161
Validation auc = 0.692488
Epoch 162
Validation auc = 0.690610
Epoch 163
Validation auc = 0.687324
Epoch 164
Loss = 4.8699e-03, PNorm = 89.1364, GNorm = 0.4162, lr_0 = 7.6436e-04
Validation auc = 0.684977
Epoch 165
Validation auc = 0.684977
Epoch 166
Validation auc = 0.683568
Epoch 167
Validation auc = 0.682160
Epoch 168
Validation auc = 0.685446
Epoch 169
Loss = 1.2016e-02, PNorm = 89.1573, GNorm = 0.0426, lr_0 = 7.5782e-04
Validation auc = 0.689671
Epoch 170
Validation auc = 0.693427
Epoch 171
Validation auc = 0.696714
Epoch 172
Validation auc = 0.691549
Epoch 173
Validation auc = 0.685446
Epoch 174
Loss = 1.1617e-01, PNorm = 89.1954, GNorm = 4.0780, lr_0 = 7.5134e-04
Validation auc = 0.689202
Epoch 175
Validation auc = 0.687793
Epoch 176
Validation auc = 0.692958
Epoch 177
Validation auc = 0.690141
Epoch 178
Validation auc = 0.694836
Epoch 179
Loss = 3.7643e-03, PNorm = 89.2414, GNorm = 0.0549, lr_0 = 7.4492e-04
Validation auc = 0.700000
Epoch 180
Validation auc = 0.697653
Epoch 181
Validation auc = 0.701878
Epoch 182
Validation auc = 0.707042
Epoch 183
Validation auc = 0.712207
Epoch 184
Loss = 3.3700e-02, PNorm = 89.2915, GNorm = 3.1275, lr_0 = 7.3855e-04
Validation auc = 0.713146
Epoch 185
Validation auc = 0.709390
Epoch 186
Validation auc = 0.701408
Epoch 187
Validation auc = 0.694836
Epoch 188
Validation auc = 0.684977
Epoch 189
Loss = 3.1340e-02, PNorm = 89.3394, GNorm = 1.6619, lr_0 = 7.3224e-04
Validation auc = 0.690141
Epoch 190
Validation auc = 0.692958
Epoch 191
Validation auc = 0.703286
Epoch 192
Validation auc = 0.705164
Epoch 193
Validation auc = 0.704695
Epoch 194
Loss = 3.8395e-03, PNorm = 89.3789, GNorm = 0.1460, lr_0 = 7.2598e-04
Validation auc = 0.700000
Epoch 195
Validation auc = 0.698122
Epoch 196
Validation auc = 0.692019
Epoch 197
Validation auc = 0.687793
Epoch 198
Validation auc = 0.684038
Epoch 199
Loss = 1.1016e-01, PNorm = 89.4203, GNorm = 3.5277, lr_0 = 7.1977e-04
Validation auc = 0.686385
Epoch 200
Validation auc = 0.688263
Epoch 201
Validation auc = 0.692488
Epoch 202
Validation auc = 0.697653
Epoch 203
Validation auc = 0.700469
Epoch 204
Loss = 2.8335e-02, PNorm = 89.4565, GNorm = 0.9185, lr_0 = 7.1362e-04
Validation auc = 0.708451
Epoch 205
Validation auc = 0.708920
Epoch 206
Validation auc = 0.702817
Epoch 207
Validation auc = 0.696714
Epoch 208
Validation auc = 0.691549
Epoch 209
Loss = 9.3126e-03, PNorm = 89.4891, GNorm = 1.3207, lr_0 = 7.0752e-04
Validation auc = 0.687324
Epoch 210
Validation auc = 0.687793
Epoch 211
Validation auc = 0.688263
Epoch 212
Validation auc = 0.687793
Epoch 213
Validation auc = 0.692019
Epoch 214
Loss = 1.6553e-02, PNorm = 89.5229, GNorm = 1.6595, lr_0 = 7.0147e-04
Validation auc = 0.695775
Epoch 215
Validation auc = 0.696714
Epoch 216
Validation auc = 0.701878
Epoch 217
Validation auc = 0.702347
Epoch 218
Validation auc = 0.702347
Epoch 219
Loss = 4.9113e-03, PNorm = 89.5488, GNorm = 0.0517, lr_0 = 6.9548e-04
Validation auc = 0.701878
Epoch 220
Validation auc = 0.700939
Epoch 221
Validation auc = 0.699531
Epoch 222
Validation auc = 0.698122
Epoch 223
Validation auc = 0.695305
Epoch 224
Loss = 1.2347e-03, PNorm = 89.5787, GNorm = 0.0243, lr_0 = 6.8953e-04
Validation auc = 0.689671
Epoch 225
Validation auc = 0.686385
Epoch 226
Validation auc = 0.680282
Epoch 227
Validation auc = 0.681221
Epoch 228
Validation auc = 0.678873
Epoch 229
Loss = 6.7567e-03, PNorm = 89.6131, GNorm = 0.3270, lr_0 = 6.8364e-04
Validation auc = 0.680751
Epoch 230
Validation auc = 0.683568
Epoch 231
Validation auc = 0.689202
Epoch 232
Validation auc = 0.692488
Epoch 233
Validation auc = 0.703756
Epoch 234
Loss = 4.2515e-02, PNorm = 89.6476, GNorm = 3.0107, lr_0 = 6.7779e-04
Validation auc = 0.703286
Epoch 235
Validation auc = 0.700469
Epoch 236
Validation auc = 0.697183
Epoch 237
Validation auc = 0.694366
Epoch 238
Validation auc = 0.689202
Epoch 239
Loss = 1.2726e-02, PNorm = 89.6876, GNorm = 0.1977, lr_0 = 6.7200e-04
Validation auc = 0.679343
Epoch 240
Validation auc = 0.677934
Epoch 241
Validation auc = 0.674178
Epoch 242
Validation auc = 0.675117
Epoch 243
Validation auc = 0.675587
Epoch 244
Loss = 4.2536e-03, PNorm = 89.7239, GNorm = 0.2353, lr_0 = 6.6625e-04
Validation auc = 0.673239
Epoch 245
Validation auc = 0.671362
Epoch 246
Validation auc = 0.673709
Epoch 247
Validation auc = 0.675587
Epoch 248
Validation auc = 0.676056
Epoch 249
Loss = 3.4669e-03, PNorm = 89.7550, GNorm = 0.2134, lr_0 = 6.6056e-04
Validation auc = 0.676995
Epoch 250
Validation auc = 0.677934
Epoch 251
Validation auc = 0.678404
Epoch 252
Validation auc = 0.679343
Epoch 253
Validation auc = 0.682629
Epoch 254
Loss = 3.6935e-03, PNorm = 89.7902, GNorm = 0.0415, lr_0 = 6.5491e-04
Validation auc = 0.687324
Epoch 255
Validation auc = 0.689202
Epoch 256
Validation auc = 0.690610
Epoch 257
Validation auc = 0.689202
Epoch 258
Validation auc = 0.691549
Epoch 259
Loss = 1.8991e-03, PNorm = 89.8337, GNorm = 0.0663, lr_0 = 6.4931e-04
Validation auc = 0.693897
Epoch 260
Validation auc = 0.692958
Epoch 261
Validation auc = 0.694366
Epoch 262
Validation auc = 0.695305
Epoch 263
Validation auc = 0.695775
Epoch 264
Loss = 2.4167e-03, PNorm = 89.8796, GNorm = 0.2257, lr_0 = 6.4376e-04
Validation auc = 0.696714
Epoch 265
Validation auc = 0.695305
Epoch 266
Validation auc = 0.693427
Epoch 267
Validation auc = 0.693427
Epoch 268
Validation auc = 0.691549
Epoch 269
Loss = 5.8656e-03, PNorm = 89.9189, GNorm = 1.3602, lr_0 = 6.3826e-04
Validation auc = 0.687324
Epoch 270
Validation auc = 0.682160
Epoch 271
Validation auc = 0.679343
Epoch 272
Validation auc = 0.676995
Epoch 273
Validation auc = 0.673709
Epoch 274
Loss = 2.3989e-03, PNorm = 89.9646, GNorm = 0.1080, lr_0 = 6.3280e-04
Validation auc = 0.676056
Epoch 275
Validation auc = 0.676056
Epoch 276
Validation auc = 0.686385
Epoch 277
Validation auc = 0.692958
Epoch 278
Validation auc = 0.698592
Epoch 279
Loss = 2.4948e-02, PNorm = 90.0184, GNorm = 1.1361, lr_0 = 6.2739e-04
Validation auc = 0.695775
Epoch 280
Validation auc = 0.694366
Epoch 281
Validation auc = 0.686854
Epoch 282
Validation auc = 0.682160
Epoch 283
Validation auc = 0.679343
Epoch 284
Loss = 5.6960e-03, PNorm = 90.0618, GNorm = 0.4288, lr_0 = 6.2203e-04
Validation auc = 0.677934
Epoch 285
Validation auc = 0.680282
Epoch 286
Validation auc = 0.683568
Epoch 287
Validation auc = 0.686385
Epoch 288
Validation auc = 0.689202
Epoch 289
Loss = 7.2572e-03, PNorm = 90.1355, GNorm = 0.6217, lr_0 = 6.1671e-04
Validation auc = 0.692019
Epoch 290
Validation auc = 0.689671
Epoch 291
Validation auc = 0.684038
Epoch 292
Validation auc = 0.684977
Epoch 293
Validation auc = 0.694366
Epoch 294
Loss = 2.6466e-02, PNorm = 90.2823, GNorm = 0.0494, lr_0 = 6.1144e-04
Validation auc = 0.695775
Epoch 295
Validation auc = 0.698592
Epoch 296
Validation auc = 0.702347
Epoch 297
Validation auc = 0.698592
Epoch 298
Validation auc = 0.690610
Epoch 299
Loss = 2.6663e-03, PNorm = 90.4470, GNorm = 0.1435, lr_0 = 6.0621e-04
Validation auc = 0.681221
Model 0 best validation auc = 0.767606 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.844070
Ensemble test auc = 0.844070
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=3400, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 17,782,601
Moving model to cuda
Epoch 0
Validation auc = 0.735681
Epoch 1
Validation auc = 0.762441
Epoch 2
Validation auc = 0.724883
Epoch 3
Validation auc = 0.681221
Epoch 4
Loss = 5.9447e-01, PNorm = 86.2021, GNorm = 13.9135, lr_0 = 6.5000e-04
Validation auc = 0.696244
Epoch 5
Validation auc = 0.691080
Epoch 6
Validation auc = 0.707042
Epoch 7
Validation auc = 0.683568
Epoch 8
Validation auc = 0.711268
Epoch 9
Loss = 2.1428e-01, PNorm = 86.4889, GNorm = 3.4521, lr_0 = 9.9743e-04
Validation auc = 0.711268
Epoch 10
Validation auc = 0.722535
Epoch 11
Validation auc = 0.714554
Epoch 12
Validation auc = 0.702347
Epoch 13
Validation auc = 0.678873
Epoch 14
Loss = 2.3725e-01, PNorm = 86.7661, GNorm = 7.5378, lr_0 = 9.8890e-04
Validation auc = 0.675587
Epoch 15
Validation auc = 0.692019
Epoch 16
Validation auc = 0.712207
Epoch 17
Validation auc = 0.702347
Epoch 18
Validation auc = 0.693427
Epoch 19
Loss = 1.5792e-01, PNorm = 86.9576, GNorm = 4.0828, lr_0 = 9.8045e-04
Validation auc = 0.684038
Epoch 20
Validation auc = 0.693897
Epoch 21
Validation auc = 0.704225
Epoch 22
Validation auc = 0.711268
Epoch 23
Validation auc = 0.714085
Epoch 24
Loss = 1.8132e-01, PNorm = 87.1022, GNorm = 8.0817, lr_0 = 9.7207e-04
Validation auc = 0.692488
Epoch 25
Validation auc = 0.677465
Epoch 26
Validation auc = 0.667606
Epoch 27
Validation auc = 0.672300
Epoch 28
Validation auc = 0.669014
Epoch 29
Loss = 1.5549e-01, PNorm = 87.2214, GNorm = 5.9167, lr_0 = 9.6376e-04
Validation auc = 0.683099
Epoch 30
Validation auc = 0.695305
Epoch 31
Validation auc = 0.698592
Epoch 32
Validation auc = 0.700000
Epoch 33
Validation auc = 0.698592
Epoch 34
Loss = 1.2587e-01, PNorm = 87.3451, GNorm = 5.4140, lr_0 = 9.5552e-04
Validation auc = 0.686854
Epoch 35
Validation auc = 0.656338
Epoch 36
Validation auc = 0.671831
Epoch 37
Validation auc = 0.673709
Epoch 38
Validation auc = 0.681221
Epoch 39
Loss = 1.5516e-01, PNorm = 87.4529, GNorm = 3.8846, lr_0 = 9.4735e-04
Validation auc = 0.674648
Epoch 40
Validation auc = 0.674648
Epoch 41
Validation auc = 0.683568
Epoch 42
Validation auc = 0.691549
Epoch 43
Validation auc = 0.694836
Epoch 44
Loss = 8.0801e-02, PNorm = 87.5588, GNorm = 2.5080, lr_0 = 9.3925e-04
Validation auc = 0.688732
Epoch 45
Validation auc = 0.688263
Epoch 46
Validation auc = 0.683099
Epoch 47
Validation auc = 0.696714
Epoch 48
Validation auc = 0.700000
Epoch 49
Loss = 7.2080e-02, PNorm = 87.6420, GNorm = 2.5511, lr_0 = 9.3122e-04
Validation auc = 0.705634
Epoch 50
Validation auc = 0.717371
Epoch 51
Validation auc = 0.722066
Epoch 52
Validation auc = 0.707981
Epoch 53
Validation auc = 0.682629
Epoch 54
Loss = 1.1445e-01, PNorm = 87.7639, GNorm = 7.6136, lr_0 = 9.2326e-04
Validation auc = 0.675587
Epoch 55
Validation auc = 0.679812
Epoch 56
Validation auc = 0.706573
Epoch 57
Validation auc = 0.703286
Epoch 58
Validation auc = 0.700939
Epoch 59
Loss = 1.3722e-01, PNorm = 87.8482, GNorm = 2.8008, lr_0 = 9.1537e-04
Validation auc = 0.715493
Epoch 60
Validation auc = 0.717371
Epoch 61
Validation auc = 0.716432
Epoch 62
Validation auc = 0.716901
Epoch 63
Validation auc = 0.715023
Epoch 64
Loss = 2.8003e-02, PNorm = 87.9353, GNorm = 0.5257, lr_0 = 9.0754e-04
Validation auc = 0.702347
Epoch 65
Validation auc = 0.702347
Epoch 66
Validation auc = 0.708451
Epoch 67
Validation auc = 0.713615
Epoch 68
Validation auc = 0.723944
Epoch 69
Loss = 1.2278e-02, PNorm = 88.0712, GNorm = 0.5543, lr_0 = 8.9978e-04
Validation auc = 0.726291
Epoch 70
Validation auc = 0.742254
Epoch 71
Validation auc = 0.738967
Epoch 72
Validation auc = 0.732394
Epoch 73
Validation auc = 0.729108
Epoch 74
Loss = 4.5053e-02, PNorm = 88.1233, GNorm = 0.5043, lr_0 = 8.9209e-04
Validation auc = 0.726291
Epoch 75
Validation auc = 0.725352
Epoch 76
Validation auc = 0.711268
Epoch 77
Validation auc = 0.692958
Epoch 78
Validation auc = 0.673709
Epoch 79
Loss = 6.2828e-02, PNorm = 88.2886, GNorm = 3.1363, lr_0 = 8.8447e-04
Validation auc = 0.670423
Epoch 80
Validation auc = 0.666197
Epoch 81
Validation auc = 0.669953
Epoch 82
Validation auc = 0.677934
Epoch 83
Validation auc = 0.683568
Epoch 84
Loss = 3.5763e-02, PNorm = 88.4082, GNorm = 1.5682, lr_0 = 8.7691e-04
Validation auc = 0.692488
Epoch 85
Validation auc = 0.698592
Epoch 86
Validation auc = 0.700000
Epoch 87
Validation auc = 0.707042
Epoch 88
Validation auc = 0.706573
Epoch 89
Loss = 9.7112e-03, PNorm = 88.4855, GNorm = 0.8843, lr_0 = 8.6941e-04
Validation auc = 0.700469
Epoch 90
Validation auc = 0.700000
Epoch 91
Validation auc = 0.700000
Epoch 92
Validation auc = 0.703286
Epoch 93
Validation auc = 0.697653
Epoch 94
Loss = 3.4047e-02, PNorm = 88.5592, GNorm = 2.3593, lr_0 = 8.6198e-04
Validation auc = 0.700939
Epoch 95
Validation auc = 0.718779
Epoch 96
Validation auc = 0.724883
Epoch 97
Validation auc = 0.728638
Epoch 98
Validation auc = 0.712207
Epoch 99
Loss = 2.2050e-02, PNorm = 88.6766, GNorm = 1.0606, lr_0 = 8.5461e-04
Validation auc = 0.692488
Epoch 100
Validation auc = 0.709859
Epoch 101
Validation auc = 0.708920
Epoch 102
Validation auc = 0.704695
Epoch 103
Validation auc = 0.700939
Epoch 104
Loss = 4.5783e-02, PNorm = 88.8265, GNorm = 4.0231, lr_0 = 8.4730e-04
Validation auc = 0.700939
Epoch 105
Validation auc = 0.700939
Epoch 106
Validation auc = 0.703286
Epoch 107
Validation auc = 0.707042
Epoch 108
Validation auc = 0.707512
Epoch 109
Loss = 3.0520e-02, PNorm = 88.9395, GNorm = 2.1702, lr_0 = 8.4006e-04
Validation auc = 0.708451
Epoch 110
Validation auc = 0.696244
Epoch 111
Validation auc = 0.694836
Epoch 112
Validation auc = 0.697183
Epoch 113
Validation auc = 0.708451
Epoch 114
Loss = 9.3878e-02, PNorm = 89.0396, GNorm = 0.6394, lr_0 = 8.3288e-04
Validation auc = 0.716432
Epoch 115
Validation auc = 0.721127
Epoch 116
Validation auc = 0.714554
Epoch 117
Validation auc = 0.725822
Epoch 118
Validation auc = 0.725822
Epoch 119
Loss = 8.1493e-03, PNorm = 89.1376, GNorm = 0.3767, lr_0 = 8.2576e-04
Validation auc = 0.717371
Epoch 120
Validation auc = 0.716432
Epoch 121
Validation auc = 0.702817
Epoch 122
Validation auc = 0.706103
Epoch 123
Validation auc = 0.705634
Epoch 124
Loss = 5.2236e-03, PNorm = 89.2135, GNorm = 0.1519, lr_0 = 8.1870e-04
Validation auc = 0.706573
Epoch 125
Validation auc = 0.702347
Epoch 126
Validation auc = 0.707042
Epoch 127
Validation auc = 0.702817
Epoch 128
Validation auc = 0.707042
Epoch 129
Loss = 1.2122e-02, PNorm = 89.2785, GNorm = 0.5301, lr_0 = 8.1170e-04
Validation auc = 0.706573
Epoch 130
Validation auc = 0.706573
Epoch 131
Validation auc = 0.703286
Epoch 132
Validation auc = 0.707042
Epoch 133
Validation auc = 0.713146
Epoch 134
Loss = 5.5489e-02, PNorm = 89.3353, GNorm = 1.7557, lr_0 = 8.0476e-04
Validation auc = 0.714085
Epoch 135
Validation auc = 0.711268
Epoch 136
Validation auc = 0.718310
Epoch 137
Validation auc = 0.723005
Epoch 138
Validation auc = 0.716901
Epoch 139
Loss = 1.8481e-01, PNorm = 89.4165, GNorm = 7.1500, lr_0 = 7.9788e-04
Validation auc = 0.706573
Epoch 140
Validation auc = 0.701878
Epoch 141
Validation auc = 0.696244
Epoch 142
Validation auc = 0.681690
Epoch 143
Validation auc = 0.690610
Epoch 144
Loss = 2.0801e-02, PNorm = 89.5262, GNorm = 1.0041, lr_0 = 7.9106e-04
Validation auc = 0.688263
Epoch 145
Validation auc = 0.702347
Epoch 146
Validation auc = 0.716432
Epoch 147
Validation auc = 0.721596
Epoch 148
Validation auc = 0.725352
Epoch 149
Loss = 1.3417e-02, PNorm = 89.6027, GNorm = 0.1740, lr_0 = 7.8430e-04
Validation auc = 0.730986
Epoch 150
Validation auc = 0.729108
Epoch 151
Validation auc = 0.724413
Epoch 152
Validation auc = 0.717840
Epoch 153
Validation auc = 0.716432
Epoch 154
Loss = 1.4843e-02, PNorm = 89.6627, GNorm = 0.9073, lr_0 = 7.7759e-04
Validation auc = 0.721127
Epoch 155
Validation auc = 0.725822
Epoch 156
Validation auc = 0.728638
Epoch 157
Validation auc = 0.735211
Epoch 158
Validation auc = 0.735681
Epoch 159
Loss = 1.2693e-02, PNorm = 89.7048, GNorm = 0.9846, lr_0 = 7.7095e-04
Validation auc = 0.735211
Epoch 160
Validation auc = 0.740376
Epoch 161
Validation auc = 0.746948
Epoch 162
Validation auc = 0.742723
Epoch 163
Validation auc = 0.738498
Epoch 164
Loss = 2.6703e-02, PNorm = 89.7538, GNorm = 1.3949, lr_0 = 7.6436e-04
Validation auc = 0.732864
Epoch 165
Validation auc = 0.726761
Epoch 166
Validation auc = 0.723474
Epoch 167
Validation auc = 0.720188
Epoch 168
Validation auc = 0.721127
Epoch 169
Loss = 8.1707e-03, PNorm = 89.8122, GNorm = 0.2411, lr_0 = 7.5782e-04
Validation auc = 0.719718
Epoch 170
Validation auc = 0.717371
Epoch 171
Validation auc = 0.718310
Epoch 172
Validation auc = 0.716432
Epoch 173
Validation auc = 0.715962
Epoch 174
Loss = 2.1442e-02, PNorm = 89.8645, GNorm = 0.3952, lr_0 = 7.5134e-04
Validation auc = 0.710798
Epoch 175
Validation auc = 0.713146
Epoch 176
Validation auc = 0.705634
Epoch 177
Validation auc = 0.706103
Epoch 178
Validation auc = 0.704695
Epoch 179
Loss = 1.9351e-02, PNorm = 89.9843, GNorm = 0.6145, lr_0 = 7.4492e-04
Validation auc = 0.706103
Epoch 180
Validation auc = 0.708451
Epoch 181
Validation auc = 0.711268
Epoch 182
Validation auc = 0.718310
Epoch 183
Validation auc = 0.722066
Epoch 184
Loss = 2.6046e-02, PNorm = 90.0730, GNorm = 3.0263, lr_0 = 7.3855e-04
Validation auc = 0.718310
Epoch 185
Validation auc = 0.716901
Epoch 186
Validation auc = 0.729108
Epoch 187
Validation auc = 0.727700
Epoch 188
Validation auc = 0.728169
Epoch 189
Loss = 1.2306e-02, PNorm = 90.1607, GNorm = 1.5879, lr_0 = 7.3224e-04
Validation auc = 0.724883
Epoch 190
Validation auc = 0.721127
Epoch 191
Validation auc = 0.738028
Epoch 192
Validation auc = 0.750704
Epoch 193
Validation auc = 0.760563
Epoch 194
Loss = 3.1156e-03, PNorm = 90.2288, GNorm = 0.0463, lr_0 = 7.2598e-04
Validation auc = 0.769484
Epoch 195
Validation auc = 0.772770
Epoch 196
Validation auc = 0.760094
Epoch 197
Validation auc = 0.754930
Epoch 198
Validation auc = 0.744601
Epoch 199
Loss = 9.5554e-04, PNorm = 90.2986, GNorm = 0.0342, lr_0 = 7.1977e-04
Validation auc = 0.737089
Epoch 200
Validation auc = 0.731925
Epoch 201
Validation auc = 0.736620
Epoch 202
Validation auc = 0.731925
Epoch 203
Validation auc = 0.730986
Epoch 204
Loss = 5.7366e-03, PNorm = 90.3860, GNorm = 0.3195, lr_0 = 7.1362e-04
Validation auc = 0.730516
Epoch 205
Validation auc = 0.729108
Epoch 206
Validation auc = 0.721596
Epoch 207
Validation auc = 0.718779
Epoch 208
Validation auc = 0.722066
Epoch 209
Loss = 1.2968e-03, PNorm = 90.4500, GNorm = 0.0265, lr_0 = 7.0752e-04
Validation auc = 0.721127
Epoch 210
Validation auc = 0.723474
Epoch 211
Validation auc = 0.721596
Epoch 212
Validation auc = 0.717840
Epoch 213
Validation auc = 0.709859
Epoch 214
Loss = 3.0463e-02, PNorm = 90.6036, GNorm = 1.7841, lr_0 = 7.0147e-04
Validation auc = 0.714554
Epoch 215
Validation auc = 0.721596
Epoch 216
Validation auc = 0.722066
Epoch 217
Validation auc = 0.721127
Epoch 218
Validation auc = 0.723944
Epoch 219
Loss = 7.8261e-03, PNorm = 90.7940, GNorm = 0.6741, lr_0 = 6.9548e-04
Validation auc = 0.727230
Epoch 220
Validation auc = 0.727700
Epoch 221
Validation auc = 0.728169
Epoch 222
Validation auc = 0.726761
Epoch 223
Validation auc = 0.726761
Epoch 224
Loss = 2.3833e-03, PNorm = 90.9411, GNorm = 0.0549, lr_0 = 6.8953e-04
Validation auc = 0.726761
Epoch 225
Validation auc = 0.728169
Epoch 226
Validation auc = 0.723005
Epoch 227
Validation auc = 0.720657
Epoch 228
Validation auc = 0.723005
Epoch 229
Loss = 1.2714e-02, PNorm = 91.0603, GNorm = 1.0141, lr_0 = 6.8364e-04
Validation auc = 0.723005
Epoch 230
Validation auc = 0.730047
Epoch 231
Validation auc = 0.733803
Epoch 232
Validation auc = 0.738028
Epoch 233
Validation auc = 0.732864
Epoch 234
Loss = 1.6246e-02, PNorm = 91.1411, GNorm = 1.0756, lr_0 = 6.7779e-04
Validation auc = 0.737559
Epoch 235
Validation auc = 0.741315
Epoch 236
Validation auc = 0.745540
Epoch 237
Validation auc = 0.747887
Epoch 238
Validation auc = 0.746009
Epoch 239
Loss = 2.9138e-03, PNorm = 91.2920, GNorm = 0.0886, lr_0 = 6.7200e-04
Validation auc = 0.743192
Epoch 240
Validation auc = 0.745070
Epoch 241
Validation auc = 0.742723
Epoch 242
Validation auc = 0.743662
Epoch 243
Validation auc = 0.743662
Epoch 244
Loss = 3.1117e-03, PNorm = 91.3915, GNorm = 0.3083, lr_0 = 6.6625e-04
Validation auc = 0.738967
Epoch 245
Validation auc = 0.736150
Epoch 246
Validation auc = 0.737559
Epoch 247
Validation auc = 0.740376
Epoch 248
Validation auc = 0.738967
Epoch 249
Loss = 8.5850e-03, PNorm = 91.4558, GNorm = 0.4299, lr_0 = 6.6056e-04
Validation auc = 0.740845
Epoch 250
Validation auc = 0.734272
Epoch 251
Validation auc = 0.732864
Epoch 252
Validation auc = 0.732394
Epoch 253
Validation auc = 0.732864
Epoch 254
Loss = 8.3198e-03, PNorm = 91.5127, GNorm = 0.3519, lr_0 = 6.5491e-04
Validation auc = 0.730516
Epoch 255
Validation auc = 0.731455
Epoch 256
Validation auc = 0.729108
Epoch 257
Validation auc = 0.732394
Epoch 258
Validation auc = 0.731925
Epoch 259
Loss = 4.2608e-03, PNorm = 91.5666, GNorm = 0.1882, lr_0 = 6.4931e-04
Validation auc = 0.730047
Epoch 260
Validation auc = 0.732864
Epoch 261
Validation auc = 0.738498
Epoch 262
Validation auc = 0.741784
Epoch 263
Validation auc = 0.748357
Epoch 264
Loss = 6.7760e-03, PNorm = 91.5998, GNorm = 0.6567, lr_0 = 6.4376e-04
Validation auc = 0.750704
Epoch 265
Validation auc = 0.752113
Epoch 266
Validation auc = 0.751643
Epoch 267
Validation auc = 0.744131
Epoch 268
Validation auc = 0.738498
Epoch 269
Loss = 2.8062e-03, PNorm = 91.6702, GNorm = 0.0613, lr_0 = 6.3826e-04
Validation auc = 0.736620
Epoch 270
Validation auc = 0.738498
Epoch 271
Validation auc = 0.741784
Epoch 272
Validation auc = 0.745540
Epoch 273
Validation auc = 0.747887
Epoch 274
Loss = 5.3698e-02, PNorm = 91.7498, GNorm = 0.0249, lr_0 = 6.3280e-04
Validation auc = 0.751643
Epoch 275
Validation auc = 0.753052
Epoch 276
Validation auc = 0.751174
Epoch 277
Validation auc = 0.753052
Epoch 278
Validation auc = 0.758685
Epoch 279
Loss = 5.6516e-02, PNorm = 91.8306, GNorm = 0.0355, lr_0 = 6.2739e-04
Validation auc = 0.757746
Epoch 280
Validation auc = 0.747887
Epoch 281
Validation auc = 0.744601
Epoch 282
Validation auc = 0.739437
Epoch 283
Validation auc = 0.736150
Epoch 284
Loss = 9.4263e-02, PNorm = 91.9153, GNorm = 3.7989, lr_0 = 6.2203e-04
Validation auc = 0.732394
Epoch 285
Validation auc = 0.741784
Epoch 286
Validation auc = 0.753052
Epoch 287
Validation auc = 0.758685
Epoch 288
Validation auc = 0.764319
Epoch 289
Loss = 7.6379e-03, PNorm = 91.9848, GNorm = 0.7751, lr_0 = 6.1671e-04
Validation auc = 0.764789
Epoch 290
Validation auc = 0.766667
Epoch 291
Validation auc = 0.761972
Epoch 292
Validation auc = 0.767606
Epoch 293
Validation auc = 0.761972
Epoch 294
Loss = 1.0611e-03, PNorm = 92.0504, GNorm = 0.0252, lr_0 = 6.1144e-04
Validation auc = 0.762441
Epoch 295
Validation auc = 0.759155
Epoch 296
Validation auc = 0.755869
Epoch 297
Validation auc = 0.754460
Epoch 298
Validation auc = 0.750704
Epoch 299
Loss = 3.4192e-03, PNorm = 92.1007, GNorm = 0.3579, lr_0 = 6.0621e-04
Validation auc = 0.753991
Model 0 best validation auc = 0.772770 on epoch 195
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.841060
Ensemble test auc = 0.841060
10-fold cross validation
	Seed 0 ==> test auc = 0.881999
	Seed 1 ==> test auc = 0.833233
	Seed 2 ==> test auc = 0.886213
	Seed 3 ==> test auc = 0.801927
	Seed 4 ==> test auc = 0.883805
	Seed 5 ==> test auc = 0.915111
	Seed 6 ==> test auc = 0.839253
	Seed 7 ==> test auc = 0.820590
	Seed 8 ==> test auc = 0.844070
	Seed 9 ==> test auc = 0.841060
Overall test auc = 0.854726 +/- 0.033439
Elapsed time = 0:05:26
